README.md in gush-3.0.0 vs README.md in gush-4.0.0

- old
+ new

@@ -157,21 +157,23 @@ Let's assume we are writing a book publishing workflow which needs to know where the PDF of the book is and under what ISBN it will be released: ```ruby class PublishBookWorkflow < Gush::Workflow - def configure(url, isbn) + def configure(url, isbn, publish: false) run FetchBook, params: { url: url } - run PublishBook, params: { book_isbn: isbn }, after: FetchBook + if publish + run PublishBook, params: { book_isbn: isbn }, after: FetchBook + end end end ``` and then create your workflow with those arguments: ```ruby -PublishBookWorkflow.create("http://url.com/book.pdf", "978-0470081204") +PublishBookWorkflow.create("http://url.com/book.pdf", "978-0470081204", publish: true) ``` and that's basically it for defining workflows, see below on how to define jobs: ## Defining jobs @@ -252,12 +254,57 @@ #=> :running|:finished|:failed ``` `reload` is needed to see the latest status, since workflows are updated asynchronously. +## Loading workflows + +### Finding a workflow by id + +``` +flow = Workflow.find(id) +``` + +### Paging through workflows + +To get workflows with pagination, use start and stop (inclusive) index values: + +``` +flows = Workflow.page(0, 99) +``` + +Or in reverse order: + +``` +flows = Workflow.page(0, 99, order: :desc) +``` + ## Advanced features +### Global parameters for jobs + +Workflows can accept a hash of `globals` that are automatically forwarded as parameters to all jobs. + +This is useful to have common functionality across workflow and job classes, such as tracking the creator id for all instances: + +```ruby +class SimpleWorkflow < Gush::Workflow + def configure(url_to_fetch_from) + run DownloadJob, params: { url: url_to_fetch_from } + end +end + +flow = SimpleWorkflow.create('http://foo.com', globals: { creator_id: 123 }) +flow.globals +=> {:creator_id=>123} +flow.jobs.first.params +=> {:creator_id=>123, :url=>"http://foo.com"} +``` + +**Note:** job params with the same key as globals will take precedence over the globals. + + ### Pipelining Gush offers a useful tool to pass results of a job to its dependencies, so they can act differently. **Example:** @@ -381,26 +428,63 @@ run AdminNotificationJob, after: notification_jobs, queue: 'admin', wait: 5.seconds end end ``` +### Customization of ActiveJob enqueueing + +There might be a case when you want to customize enqueing a job with more than just the above two options (`queue` and `wait`). + +To pass additional options to `ActiveJob.set`, override `Job#worker_options`, e.g.: + +```ruby + +class ScheduledJob < Gush::Job + + def worker_options + super.merge(wait_until: Time.at(params[:start_at])) + end + +end +``` + +Or to entirely customize the ActiveJob integration, override `Job#enqueue_worker!`, e.g.: + +```ruby + +class SynchronousJob < Gush::Job + + def enqueue_worker!(options = {}) + Gush::Worker.perform_now(workflow_id, name) + end + +end +``` + + ## Command line interface (CLI) ### Checking status - of a specific workflow: ``` bundle exec gush show <workflow_id> ``` -- of all created workflows: +- of a page of workflows: ``` bundle exec gush list ``` +- of the most recent 100 workflows + + ``` + bundle exec gush list -99 -1 + ``` + ### Vizualizing workflows as image This requires that you have imagemagick installed on your computer: @@ -422,23 +506,27 @@ end ``` ### Cleaning up afterwards -Running `NotifyWorkflow.create` inserts multiple keys into Redis every time it is ran. This data might be useful for analysis but at a certain point it can be purged via Redis TTL. By default gush and Redis will keep keys forever. To configure expiration you need to 2 things. Create initializer (specify config.ttl in seconds, be different per environment). +Running `NotifyWorkflow.create` inserts multiple keys into Redis every time it is run. This data might be useful for analysis but at a certain point it can be purged. By default gush and Redis will keep keys forever. To configure expiration you need to do two things. +1. Create an initializer that specifies `config.ttl` in seconds. Best NOT to set TTL to be too short (like minutes) but about a week in length. + ```ruby # config/initializers/gush.rb Gush.configure do |config| config.redis_url = "redis://localhost:6379" config.concurrency = 5 config.ttl = 3600*24*7 end ``` -And you need to call `flow.expire!` (optionally passing custom TTL value overriding `config.ttl`). This gives you control whether to expire data for specific workflow. Best NOT to set TTL to be too short (like minutes) but about a week in length. And you can run `Client.expire_workflow` and `Client.expire_job` passing appropriate IDs and TTL (pass -1 to NOT expire) values. +2. Call `Client#expire_workflows` periodically, which will clear all expired stored workflow and job data and indexes. This method can be called at any rate, but ideally should be called at least once for every 1000 workflows created. +If you need more control over individual workflow expiration, you can call `flow.expire!(ttl)` with a TTL different from the Gush configuration, or with -1 to never expire the workflow. + ### Avoid overlapping workflows Since we do not know how long our workflow execution will take we might want to avoid starting the next scheduled workflow iteration while the current one with same class is still running. Long term this could be moved into core library, perhaps `Workflow.find_by_class(klass)` ```ruby @@ -450,9 +538,22 @@ return true if flow.to_hash[:name] == klass && flow.running? end return false end ``` + +## Gush 3.0 Migration + +Gush 3.0 adds indexing for fast workflow pagination and changes the mechanism for expiring workflow data from Redis. + +### Migration + +Run `bundle exec gush migrate` after upgrading. This will update internal data structures. + +### Expiration API + +Periodically run `Gush::Client.new.expire_workflows` to expire data. Workflows will be automatically enrolled in this expiration, so there is no longer a need to call `workflow.expire!`. + ## Contributors - [Mateusz Lenik](https://github.com/mlen) - [Michał Krzyżanowski](https://github.com/krzyzak)