README.md in knapsack_pro-0.25.0 vs README.md in knapsack_pro-0.26.0

- old
+ new

@@ -94,10 +94,12 @@ - [Info for semaphoreapp.com users](#info-for-semaphoreappcom-users) - [Info for buildkite.com users](#info-for-buildkitecom-users) - [Info for snap-ci.com users](#info-for-snap-cicom-users) - [FAQ](#faq) - [How to run tests for particular CI node in your development environment](#how-to-run-tests-for-particular-ci-node-in-your-development-environment) + - [for knapack_pro regular mode](#for-knapack_pro-regular-mode) + - [for knapsack_pro queue mode](#for-knapsack_pro-queue-mode) - [What happens when Knapsack Pro API is not available/not reachable temporarily?](#what-happens-when-knapsack-pro-api-is-not-availablenot-reachable-temporarily) - [How can I change log level?](#how-can-i-change-log-level) - [How to split tests based on test level instead of test file level?](#how-to-split-tests-based-on-test-level-instead-of-test-file-level) - [A. Create multiple small test files](#a-create-multiple-small-test-files) - [B. Use tags to mark set of tests in particular test file](#b-use-tags-to-mark-set-of-tests-in-particular-test-file) @@ -143,12 +145,14 @@ You can read next section only if you want to better understand optional gem configuration and features. ## How to set up -If you are using [VCR gem](https://github.com/vcr/vcr) then add Knapsack Pro API subdomain to [ignore hosts](https://www.relishapp.com/vcr/vcr/v/2-9-3/docs/configuration/ignore-request): +If you use [VCR](https://github.com/vcr/vcr), [WebMock](https://github.com/bblimke/webmock) or [FakeWeb](https://github.com/chrisk/fakeweb) gems then you need to allow them to make requests to Knapsack Pro API. +For VCR add Knapsack Pro API subdomain to [ignore hosts](https://www.relishapp.com/vcr/vcr/v/2-9-3/docs/configuration/ignore-request): + ```ruby # spec/spec_helper.rb or wherever is your VCR configuration require 'vcr' VCR.configure do |config| @@ -157,20 +161,25 @@ end # add below when you hook into webmock require 'webmock/rspec' WebMock.disable_net_connect!(:allow => ['api.knapsackpro.com']) + +# add below when you use FakeWeb +require 'fakeweb' +FakeWeb.allow_net_connect = %r[^https?://api\.knapsackpro\.com] ``` Ensure you have require false for webmock gem when VCR is hook into it. Thanks to that webmock configuration in `spec_helper.rb` is loaded properly. ```ruby # Gemfile group :test do gem 'vcr' gem 'webmock', require: false + gem 'fakeweb', require: false # example when you use fakeweb end ``` ### Usage (How to set up 1 of 3) @@ -640,10 +649,12 @@ ## FAQ ### How to run tests for particular CI node in your development environment +#### for knapack_pro regular mode + In your development environment you can debug tests that were run on the particular CI node. For instance to run subset of tests for the first CI node with specified seed you can do. KNAPSACK_PRO_TEST_SUITE_TOKEN_RSPEC=token \ KNAPSACK_PRO_REPOSITORY_ADAPTER=git \ @@ -651,9 +662,54 @@ KNAPSACK_PRO_CI_NODE_TOTAL=2 \ KNAPSACK_PRO_CI_NODE_INDEX=0 \ bundle exec rake "knapsack_pro:rspec[--seed 123]" Above example is for RSpec. You can use respectively rake task name and token environment variable when you want to run tests for minitest, cucumber or spinach. +It should work when all CI nodes finished work and sent time execution data to Knapsack Pro API. +You can visit [user dashboard](https://knapsackpro.com/dashboard) to preview particular CI build and ensure time execution data were collected from all CI nodes. +If at least one CI node has not sent time execution data to the Knapsack Pro API then you should check below solution. + +Check test runner output on particular CI node you would like to retry in development. You should see at the beginning of rspec command an output that can +be copied and executed in development. + + /Users/ubuntu/.rvm/gems/ruby-2.4.0/gems/rspec-core-3.4.4/exe/rspec spec/foo_spec.rb spec/bar_spec.rb --default-path spec + +Command similar to above can be executed in your development this way: + + bundle exec rspec spec/foo_spec.rb spec/bar_spec.rb --default-path spec + +If you were running your tests with `--order random` on your CI then you can additionaly pass seed param with proper value in above command (`--seed 123`). + +#### for knapsack_pro queue mode + +To retry the particular CI node do this on your machine: + + KNAPSACK_PRO_TEST_SUITE_TOKEN_RSPEC=token \ + KNAPSACK_PRO_REPOSITORY_ADAPTER=git \ + KNAPSACK_PRO_PROJECT_DIR=~/projects/rails-app \ + KNAPSACK_PRO_CI_NODE_TOTAL=2 \ + KNAPSACK_PRO_CI_NODE_INDEX=0 \ + KNAPSACK_PRO_FIXED_QUEUE_SPLIT=true \ + bundle exec rake "knapsack_pro:queue:rspec" + +If you were running your tests with `--order random` on your CI like this: + + bundle exec rake "knapsack_pro:queue:rspec[--order random]" + +Then you can find the seed number visible in rspec output: + + (...) + Randomized with seed 123 + +You can pass the seed in your local environment to reproduce the tests in the same order as they were executed on CI node: + + KNAPSACK_PRO_TEST_SUITE_TOKEN_RSPEC=token \ + KNAPSACK_PRO_REPOSITORY_ADAPTER=git \ + KNAPSACK_PRO_PROJECT_DIR=~/projects/rails-app \ + KNAPSACK_PRO_CI_NODE_TOTAL=2 \ + KNAPSACK_PRO_CI_NODE_INDEX=0 \ + KNAPSACK_PRO_FIXED_QUEUE_SPLIT=true \ + bundle exec rake "knapsack_pro:queue:rspec[--seed 123]" ### What happens when Knapsack Pro API is not available/not reachable temporarily? knapsack_pro gem has fallback behaviour and it will split test files across CI nodes based on popular test directory names.