README.md in knapsack_pro-1.18.2 vs README.md in knapsack_pro-1.19.0
- old
+ new
@@ -80,10 +80,12 @@
- [Additional info about queue mode](#additional-info-about-queue-mode)
- [Extra configuration for Queue Mode](#extra-configuration-for-queue-mode)
- [KNAPSACK_PRO_FIXED_QUEUE_SPLIT (remember queue split on retry CI node)](#knapsack_pro_fixed_queue_split-remember-queue-split-on-retry-ci-node)
- [KNAPSACK_PRO_MODIFY_DEFAULT_RSPEC_FORMATTERS (hide duplicated summary of pending and failed tests)](#knapsack_pro_modify_default_rspec_formatters-hide-duplicated-summary-of-pending-and-failed-tests)
- [Supported test runners in queue mode](#supported-test-runners-in-queue-mode)
+- [Split test files by test cases](#split-test-files-by-test-cases)
+ - [RSpec split test files by test examples (individual `it`)](#rspec-split-test-files-by-test-examples-individual-it)
- [Extra configuration for CI server](#extra-configuration-for-ci-server)
- [Info about ENV variables](#info-about-env-variables)
- [KNAPSACK_PRO_FIXED_TEST_SUITE_SPLIT (test suite split based on seed)](#knapsack_pro_fixed_test_suite_split-test-suite-split-based-on-seed)
- [Environment variables for debugging gem](#environment-variables-for-debugging-gem)
- [Required CI configuration if you use retry single failed CI node feature on your CI server when KNAPSACK_PRO_FIXED_QUEUE_SPLIT=true (in Queue Mode) or KNAPSACK_PRO_FIXED_TEST_SUITE_SPLIT=true (in Regular Mode)](#required-ci-configuration-if-you-use-retry-single-failed-ci-node-feature-on-your-ci-server-when-knapsack_pro_fixed_queue_splittrue-in-queue-mode-or-knapsack_pro_fixed_test_suite_splittrue-in-regular-mode)
@@ -95,10 +97,11 @@
- [Passing arguments to spinach](#passing-arguments-to-spinach)
- [Knapsack Pro binary](#knapsack-pro-binary)
- [Test file names encryption](#test-file-names-encryption)
- [How to enable test file names encryption?](#how-to-enable-test-file-names-encryption)
- [How to debug test file names?](#how-to-debug-test-file-names)
+ - [Preview encrypted RSpec test example paths?](#preview-encrypted-rspec-test-example-paths)
- [How to enable branch names encryption?](#how-to-enable-branch-names-encryption)
- [How to debug branch names?](#how-to-debug-branch-names)
- [Supported CI providers](#supported-ci-providers)
- [Info for CircleCI users](#info-for-circleci-users)
- [CircleCI and knapsack_pro Queue Mode](#circleci-and-knapsack_pro-queue-mode)
@@ -590,10 +593,37 @@
* RSpec
* Minitest
* Cucumber
+## Split test files by test cases
+
+__Note:__ this is an experimental feature. It works for Regular Mode and Queue Mode. For large test suite with a few thousand test files, it may generate too many RSpec test example paths that may lead to too large JSON payload in request to Knapsack Pro API and this could trigger the API timeout.
+
+Please give us feedback so we could improve the feature.
+https://knapsackpro.com/contact
+
+__How it works__: You can split slow test file by test cases. Thanks to that the test file can be split across parallel CI nodes because test cases from the test file will run on different CI nodes.
+
+This is helpful when you have one or a few very slow test files that are a bottleneck for CI build speed and you don't want to manually create a few smaller test files from the slow test files. Instead, you can tell `knapsack_pro` gem to split your test files by test cases across parallel CI nodes.
+
+### RSpec split test files by test examples (by individual `it`s)
+
+In order to split RSpec test files by test examples across parallel CI nodes you need to set flag:
+
+```
+KNAPSACK_PRO_RSPEC_SPLIT_BY_TEST_EXAMPLES=true
+```
+
+Thanks to that your CI build speed can be faster. We recommend using this feature with Queue Mode to ensure parallel CI nodes finish work at a similar time which gives you the shortest CI build time.
+
+Doing tests split by test examples can generate a lot of logs by `knapsack_pro` gem in Queue Mode. We recommend to set log level to:
+
+```
+KNAPSACK_PRO_LOG_LEVEL=warn
+```
+
## Extra configuration for CI server
### Info about ENV variables
By default knapsack_pro gem [supports a few CI providers](#supported-ci-providers) so you don't need to set some environment variables.
@@ -779,10 +809,20 @@
KNAPSACK_PRO_SALT=xxx bundle exec rake knapsack_pro:encrypted_test_file_names[rspec]
```
You can pass the name of test runner like `rspec`, `minitest`, `test_unit`, `cucumber`, `spinach` as argument to rake task.
+##### Preview encrypted RSpec test example paths?
+
+If you split RSpec tests by test examples (by individual `it`) you can preview encrypted test example paths this way:
+
+```bash
+KNAPSACK_PRO_RSPEC_SPLIT_BY_TEST_EXAMPLES=true \
+KNAPSACK_PRO_SALT=xxx \
+bundle exec rake knapsack_pro:encrypted_test_file_names[rspec]
+```
+
#### How to enable branch names encryption?
You need to add environment variable `KNAPSACK_PRO_BRANCH_ENCRYPTED=true` to your CI server.
Note: there are a few branch names that won't be encrypted because we use them as fallback branches on Knapsack Pro API side to determine time execution for test files during split for newly created branches.
@@ -2314,10 +2354,10 @@
Now you can preview logs in `Artifacts` tab in the Circle CI build view.
#### How to split tests based on test level instead of test file level?
-If you want to split one big test file (test file with long time execution) across multiple CI nodes then you can:
+If you want to split one big test file (test file with long time execution) across multiple CI nodes then you can [check this tip](#split-test-files-by-test-cases) or use other methods like:
##### A. Create multiple small test files
Create multiple small test files instead of one long running test file with many test cases.
A lot of small test files will give you better test suite split results.