README.md in backburner-0.2.5 vs README.md in backburner-0.2.6

- old
+ new

@@ -45,20 +45,22 @@ A single instance of Beanstalk is perfectly capable of handling thousands of jobs a second (or more, depending on your job size) because it is an in-memory, event-driven system. Powered by libevent under the hood, it requires zero setup (launch and forget, à la memcached), optional log based persistence, an easily parsed ASCII protocol, and a rich set of tools for job management that go well beyond a simple FIFO work queue. -Beanstalk supports the following features natively, out of the box, without any questions asked: +Beanstalkd supports the following features out of the box: - * **Parallel Queues** - Supports multiple work queues created on demand. - * **Reliable** - Beanstalk’s reserve, work, delete cycle ensures reliable processing. - * **Scheduling** - Delay enqueuing jobs by a specified interval to schedule processing later. - * **Fast** - Processes thousands of jobs per second; **significantly** [faster than alternatives](http://adam.heroku.com/past/2010/4/24/beanstalk_a_simple_and_fast_queueing_backend). - * **Priorities** - Specify priority so important jobs can be processed quickly. - * **Persistence** - Jobs are stored in memory for speed, but logged to disk for safe keeping. - * **Federation** - Horizontal scalability provided through federation by the client. - * **Error Handling** - Bury any job which causes an error for later debugging and inspection. +| Feature | Description | +| ------- | ------------------------------- | +| **Parallelized** | Supports multiple work queues created on demand. | +| **Reliable** | Beanstalk’s reserve, work, delete cycle ensures reliable processing. | +| **Scheduling** | Delay enqueuing jobs by a specified interval to schedule processing later | +| **Fast** | Processes thousands of jobs per second without breaking a sweat. | +| **Priorities** | Specify priority so important jobs can be processed quickly. | +| **Persistence** | Jobs are stored in memory for speed, but logged to disk for safe keeping. | +| **Federation** | Horizontal scalability provided through federation by the client. | +| **Error Handling** | Bury any job which causes an error for later debugging and inspection.| Keep in mind that these features are supported out of the box with beanstalk and require no special code within this gem to support. In the end, **beanstalk is the ideal job queue** while also being ridiculously easy to install and setup. ## Installation @@ -98,17 +100,19 @@ end ``` The key options available are: - * The `beanstalk_url` supports a string such as 'beanstalk://127.0.0.1' or an array of addresses. - * The `tube_namespace` is the prefix used for all tubes related to this backburner queue. - * The `on_error` is a callback that gets invoked with the error whenever any job in the system fails. - * The `default_worker` is the processing worker that will be used if no other worker is specified. - * The `max_job_retries` determines how many times to retry a job before burying - * The `retry_delay` determines the base time to wait (in secs) between retries - * The `logger` is the logger object written to when backburner wants to report info or errors. +| Option | Description | +| ------- | ------------------------------- | +| `beanstalk_url` | Address such as 'beanstalk://127.0.0.1' or an array of addresses. | +| `tube_namespace` | Prefix used for all tubes related to this backburner queue. | +| `on_error` | Lambda invoked with the error whenever any job in the system fails. | +| `default_worker` | Worker class that will be used if no other worker is specified. | +| `max_job_retries`| Integer defines how many times to retry a job before burying. | +| `retry_delay` | Integer defines the base time to wait (in secs) between job retries. | +| `logger` | Logger recorded to when backburner wants to report info or errors. | ## Usage Backburner allows you to create jobs and place them on a beanstalk queue, and later pull those jobs off the queue and process them asynchronously. @@ -208,10 +212,48 @@ bundle exec backburner newsletter-sender,push-message -d -P /var/run/backburner.pid -l /var/log/backburner.log ``` This will daemonize the worker and store the pid and logs automatically. +### Persistence + +Jobs are persisted to queues as JSON objects. Let's take our `User` +example from above. We'll run the following code to create a job: + +``` ruby +User.async.reset_password(@user.id) +``` + +The following JSON will be stored in the `{namespace}.user` queue: + +``` javascript +{ + 'class': 'User', + 'args': [nil, 'reset_password', 123] +} +``` + +The first argument is the 'id' of the object in the case of an instance method being async'ed. For example: + +```ruby +@device = Device.find(987) +@user = User.find(246) +@user.async.activate(@device.id) +``` + +would be stored as: + +``` javascript +{ + 'class': 'User', + 'args': [246, 'activate', 987] +} +``` + +Since all jobs are persisted in JSON, your jobs must only accept arguments that can be encoded into that format. +This is why our examples use object IDs instead of passing around objects. + ### Processing Strategies In Backburner, there are actually multiple different strategies for processing jobs which are reflected by multiple workers. Custom workers can be [defined fairly easily](https://github.com/nesquena/backburner/wiki/Defining-Workers). @@ -265,18 +307,10 @@ Backburner.default_queues << NewsletterJob.queue ``` The `default_queues` stores the specific list of queues that should be processed by default by a worker. -### Hooks - -Backburner is highly extensible and can be tailored to your needs by using various hooks that -can be triggered across the job processing lifecycle. -Often using hooks is much easier then trying to monkey patch the externals. - -Check out [HOOKS.md](https://github.com/nesquena/backburner/blob/master/HOOKS.md) for a detailed overview on using hooks. - ### Failures When a job fails in backburner (usually because an exception was raised), the job will be released and retried again (with progressive delays in between) until the `max_job_retries` configuration is reached. @@ -314,16 +348,18 @@ end ``` Be sure to check logs whenever things do not seem to be processing. -### Web Front-end +### Hooks -Be sure to check out the Sinatra-powered project [beanstalkd_view](https://github.com/denniskuczynski/beanstalkd_view) -by [denniskuczynski](http://github.com/denniskuczynski) which provides an excellent overview of the tubes and -jobs processed by your beanstalk workers. An excellent addition to your Backburner setup. +Backburner is highly extensible and can be tailored to your needs by using various hooks that +can be triggered across the job processing lifecycle. +Often using hooks is much easier then trying to monkey patch the externals. +Check out [HOOKS.md](https://github.com/nesquena/backburner/blob/master/HOOKS.md) for a detailed overview on using hooks. + ### Workers in Production Once you have Backburner setup in your application, starting workers is really easy. Once [beanstalkd](http://kr.github.com/beanstalkd/download.html) is installed, your best bet is to use the built-in rake task that comes with Backburner. Simply add the task to your Rakefile: @@ -340,9 +376,15 @@ ``` The best way to deploy these rake tasks is using a monitoring library. We suggest [God](https://github.com/mojombo/god/) which watches processes and ensures their stability. A simple God recipe for Backburner can be found in [examples/god](https://github.com/nesquena/backburner/blob/master/examples/god.rb). + +### Web Front-end + +Be sure to check out the Sinatra-powered project [beanstalkd_view](https://github.com/denniskuczynski/beanstalkd_view) +by [denniskuczynski](http://github.com/denniskuczynski) which provides an excellent overview of the tubes and +jobs processed by your beanstalk workers. An excellent addition to your Backburner setup. ## Acknowledgements * [Nathan Esquenazi](https://github.com/nesquena) - Project maintainer * Kristen Tucker - Coming up with the gem name \ No newline at end of file