README.md in knapsack_pro-1.12.0 vs README.md in knapsack_pro-1.12.1

- old
+ new

@@ -119,10 +119,11 @@ - [Why my CI build fails when I use Test::Unit even when all tests passed?](#why-my-ci-build-fails-when-i-use-testunit-even-when-all-tests-passed) - [Why I see HEAD as branch name in user dashboard for Build metrics for my API token?](#why-i-see-head-as-branch-name-in-user-dashboard-for-build-metrics-for-my-api-token) - [Why Capybara feature tests randomly fail when using CI parallelisation?](#why-capybara-feature-tests-randomly-fail-when-using-ci-parallelisation) - [Why knapsack_pro freezes / hangs my CI (for instance Travis)?](#why-knapsack_pro-freezes--hangs-my-ci-for-instance-travis) - [Why tests hitting external API fail?](#why-tests-hitting-external-api-fail) + - [Why green test suite for Cucumber 2.99 tests always fails with `invalid option: --require`?](#why-green-test-suite-for-cucumber-299-tests-always-fails-with-invalid-option---require) - [Queue Mode problems](#queue-mode-problems) - [Why when I use Queue Mode for RSpec then my tests fail?](#why-when-i-use-queue-mode-for-rspec-then-my-tests-fail) - [Why when I use Queue Mode for RSpec then FactoryBot/FactoryGirl tests fail?](#why-when-i-use-queue-mode-for-rspec-then-factorybotfactorygirl-tests-fail) - [Why when I use Queue Mode for RSpec then my rake tasks are run twice?](#why-when-i-use-queue-mode-for-rspec-then-my-rake-tasks-are-run-twice) - [Why when I use Queue Mode for RSpec then I see error `superclass mismatch for class`?](#why-when-i-use-queue-mode-for-rspec-then-i-see-error-superclass-mismatch-for-class) @@ -1457,10 +1458,52 @@ To fix that you can think of: * using [VCR](https://github.com/vcr/vcr) gem to record HTTP requests and then instead of doing real HTTP requests just reply recorded requests. * maybe you could write your tests in a way when you generate some fake customers or subscriptions with fake id and each test has different customer id so there will be no conflict when 2 tests are run at the same time. +#### Why green test suite for Cucumber 2.99 tests always fails with `invalid option: --require`? + +If you use old Cucumber version 2.99 and `cucumber-rails` gem you could notice bug that knapsack_pro for Cucumber fails with `1` exit status. Error you may see: + +``` +invalid option: --require + +minitest options: + -h, --help Display this help. + --no-plugins Bypass minitest plugin auto-loading (or set $MT_NO_PLUGINS). + -s, --seed SEED Sets random seed. Also via env. Eg: SEED=n rake + -v, --verbose Verbose. Show progress processing files. + -n, --name PATTERN Filter run on /regexp/ or string. + --exclude PATTERN Exclude /regexp/ or string from run. + +Known extensions: rails, pride + -w, --warnings Run with Ruby warnings enabled + -e, --environment ENV Run tests in the ENV environment + -b, --backtrace Show the complete backtrace + -d, --defer-output Output test failures and errors after the test run + -f, --fail-fast Abort test run on first failure or error + -c, --[no-]color Enable color in the output + -p, --pride Pride. Show your testing pride! + +# exit status is 1 - which means failed tests +> echo $? +1 +``` + +The root problem is that Rails add `minitest` gem and it is started when `cucumber/rails` is loaded. It should not be. You can fix it by adding below in file `features/support/env.rb`: + +```ruby +# features/support/env.rb +require 'cucumber/rails' + +# this must be after we require cucumber/rails +require 'multi_test' +MultiTest.disable_autorun +``` + +The solution comes from: [cucumber/multi_test](https://github.com/cucumber/multi_test/pull/2#issuecomment-21863459) + #### Queue Mode problems ##### Why when I use Queue Mode for RSpec then my tests fail? knapsack_pro Queue Mode uses `RSpec::Core::Runner` feature that allows [running specs multiple times with different runner options in the same process](https://relishapp.com/rspec/rspec-core/docs/running-specs-multiple-times-with-different-runner-options-in-the-same-process). @@ -1599,12 +1642,12 @@ ##### Why all test files have 0.1s time execution for my CI build in user dashboard? If you go to [user dashboard](https://knapsackpro.com/dashboard) and open `Build metrics` for your API token and you open CI build for your last git commit you should see there info about collected time execution data from all CI nodes. If you see all test files have 0.1s time execution then please ensure: -* you don't clean up `tmp` directory in your tests (for instance in RSpec hooks like `before` or `after`) so knapsack_pro can publish measured time execution data to Knapsack Pro API server. knapsack_pro Queue Mode saves temporary files with collected time execution data in `your_rails_project/tmp/knapsack_pro/queue/`. -* please ensure you have in your `rails_helper.rb` or `spec_helper.rb` line: +* you should not clean up `tmp` directory in your tests (for instance in RSpec hooks like `before` or `after`) so knapsack_pro can publish measured time execution data to Knapsack Pro API server. knapsack_pro Queue Mode saves temporary files with collected time execution data in `your_rails_project/tmp/knapsack_pro/queue/`. +* please ensure you have in your `rails_helper.rb` or `spec_helper.rb` line that allows to measure tests: ```ruby require 'knapsack_pro' # CUSTOM_CONFIG_GOES_HERE @@ -2544,9 +2587,33 @@ We decrease the number of CI node total by 1 that knapsack_pro can see. This way you can run tests with knapsack_pro only on two CI nodes. On the 3rd CI node, you can run other things like linters etc. If you would like to check what is the CI node total ENV variable name exposed by your CI provider you can check that in your CI provider environment variables docs or preview the [ENV variables that knapsack_pro can read](https://github.com/KnapsackPro/knapsack_pro-ruby/tree/master/lib/knapsack_pro/config/ci) for supported CI providers. + +If you use for instance Heroku CI that allows you to provide only one test command you can make a bash script to control what's executed on particular CI node: + +```bash +#!/bin/bash +# add this file in bin/knapsack_pro_rspec_and_npm_test and change chmod +# $ chmod a+x bin/knapsack_pro_rspec_and_npm_test + +# 15 is last CI node (index starts from 0, so in total we have 16 parallel Heroku dynos) +if [ "$CI_NODE_TOTAL" == "15" ]; then + # run npm tests on the last CI node + npm test +else + KNAPSACK_PRO_CI_NODE_TOTAL=$((CI_NODE_TOTAL-1)) bundle exec rake knapsack_pro:queue:rspec +fi +``` + +then in your Heroku CI config `app.json` set: + +``` +"scripts": { + "test": "bin/knapsack_pro_rspec_and_npm_test" +} +``` #### How to use CodeClimate with knapsack_pro? You can check articles about CodeClimate configuration with knapsack_pro gem: * [CodeClimate and CircleCI 2.0 parallel builds config for RSpec with SimpleCov and JUnit formatter](https://docs.knapsackpro.com/2019/codeclimate-and-circleci-2-0-parallel-builds-config-for-rspec-with-simplecov-and-junit-formatter)