README.md in rocketjob-6.0.0.rc2 vs README.md in rocketjob-6.0.0.rc3
- old
+ new
@@ -15,24 +15,154 @@
## Support
* Questions? Join the chat room on Gitter for [rocketjob support](https://gitter.im/rocketjob/support)
* [Report bugs](https://github.com/rocketjob/rocketjob/issues)
-## Rocket Job v5
+## Rocket Job v6
- Support for Ruby v3 and Rails 6.
-- Multiple output file support through extended `output_categories` capability.
- - File output formats for each category. For example: CSV, PSV, JSON, etc.
-- Support for AWS DocumentDB as the data store.
+- Major enhancements in Batch job support:
+ - Direct built-in Tabular support for all input and output categories.
+ - Multiple output file support, each with its own settings for:
+ - Compression
+ - GZip, Zip, BZip2 (Chunked for much faster loading into Apache Spark).
+ - Encryption
+ - PGP, Symmetric Encryption.
+ - File format
+ - CSV, PSV, JSON, Fixed Format, xlsx.
+- Significant error handling improvements, especially around throttle failures
+ that used to result in "hanging" jobs.
+- Support AWS DocumentDB in addition to MongoDB as the data store.
- Removed use of Symbols to meet Symbol deprecation in MongoDB and Mongoid.
-The following plugins have been deprecated and will be removed in Rocket Job v5.1
-- RocketJob::Batch::Tabular::Input
-- RocketJob::Batch::Tabular::Output
+### Upgrading to Rocket Job v6
+The following plugins have been deprecated and are no longer loaded by default.
+- `RocketJob::Batch::Tabular::Input`
+- `RocketJob::Batch::Tabular::Output`
+
+If your code relies on these plugins and you still want to upgrade to Rocket Job v6,
+add the following require statement to any jobs that still use them:
+
+~~~ruby
+require "rocket_job/batch/tabular"
+~~~
+
+It is important to migrate away from these plugins, since they will be removed in a future release.
+
+#### Upgrading Batch Jobs to Rocket Job v6
+
+Rocket Job v6 replaces the array of symbol type for `input_categories` and `output_categories`
+with an array of `RocketJob::Category::Input` and `RocketJob::Category::Output`.
+
+Jobs that added or modified the input or output categories need to be upgraded. For example:
+~~~ruby
+class MyJob < RocketJob::Job
+ include RocketJob::Batch
+
+ self.output_categories = [:main, :errors, :ignored]
+end
+~~~
+
+Needs to be changed to:
+~~~ruby
+class MyJob < RocketJob::Job
+ include RocketJob::Batch
+
+ output_category name: :main
+ output_category name: :errors
+ output_category name: :ignored
+end
+~~~
+
+##### slice_size, encrypt, compress
+
+These fields have been removed from the job itself:
+~~~ruby
+class MyJob < RocketJob::Job
+ include RocketJob::Batch
+
+ self.slice_sice = 1_000
+ self.encrypt = true
+ self.compress = true
+end
+~~~
+
+They are now specified on the `input_category` as follows:
+- `slice_size` just moves under `input_category`.
+- `encrypt` becomes an option to `serializer`.
+- `compress` is now the default for all batch jobs so is not needed.
+
+If the serializer is set to `encrypt` then it is automatically compressed.
+
+~~~ruby
+class MyJob < RocketJob::Job
+ include RocketJob::Batch
+
+ input_category slice_sice: 1_000, serializer: :encrypt
+end
+~~~
+
+##### collect_output, collect_nil_output
+
+The following fields have been moved from the job itself:
+~~~ruby
+class MyJob < RocketJob::Job
+ include RocketJob::Batch
+
+ self.collect_output = true
+ self.collect_nil_output = true
+end
+~~~
+
+Into the corresponding `output_category`:
+- `collect_output` no longer has any meaning. Output is collected anytime an `output_category` is defined.
+- `collect_nil_output` is now the option `nils` on the `output_category.
+ It defaults to `false` so that by default any `nil` output from the `perform` method is not collected.
+~~~ruby
+class MyJob < RocketJob::Job
+ include RocketJob::Batch
+
+ output_category nils: true
+end
+~~~
+
+##### name
+
+For both `input_category` and `output_category`, when the `name` argument is not supplied
+it defaults to `:main`.
+
+For Example:
+~~~ruby
+class MyJob < RocketJob::Job
+ include RocketJob::Batch
+
+ input_category name: :main, serializer: :encrypt
+ output_category name: :main
+end
+~~~
+
+Is the same as:
+~~~ruby
+class MyJob < RocketJob::Job
+ include RocketJob::Batch
+
+ input_category serializer: :encrypt
+ output_category
+end
+~~~
+
+##### Existing and inflight jobs
+
+When migrating to Rocket Job 6, it is recommended to load every job and then save it back again as part of the
+deployment. When the job loads it will automatically convert itself from the old schema to the new v6 schema.
+
+In flight jobs should not be affected, other than it is important to shutdown all running batch
+servers _before_ running any new instances.
+
## Rocket Job v4
-Rocket Job Pro is now open source and included in Rocket Job.
+Rocket Job Pro is now fully open source and included in Rocket Job under the Apache License.
The `RocketJob::Batch` plugin now adds batch processing capabilities to break up a single task into many
concurrent workers processing slices of the entire job at the same time.