README.md in parallel_minion-0.0.1 vs README.md in parallel_minion-0.1.0
- old
+ new
@@ -1,51 +1,69 @@
parallel_minion
===============
-Parallel Minion supports easily handing work off to Minions (Threads) so that tasks
+Parallel Minion supports easily handing work off to minions (threads) so that tasks
that would normally be performed sequentially can easily be executed in parallel.
This allows Ruby and Rails applications to very easily do many tasks at the same
time so that results are returned more quickly.
-Our use-case for Minions is where an application grew to a point where it would
+Our use-case for minions is where an application grew to a point where it would
be useful to run some of the steps in fulfilling a single request in parallel.
## Features:
Exceptions
-- Any exceptions raised in Minions are captured and propagated back to the
+- Any exceptions raised in minions are captured and propagated back to the
calling thread when #result is called
-- Makes exception handling simple with a drop-in replacement in existing code
+- Makes exception handling simple with a drop-in replacement for existing code
+- Avoids having to implement more complex actors and supervisors required
+ by some concurrency frameworks
Timeouts
-- Timeout when a Minion does not return within a specified time
-- Timeouts are a useful feature when one of the Minions fails to respond in a
+- Timeout when a minion does not return within a specified time
+- Timeouts are a useful feature when one of the minions fails to respond in a
reasonable amount of time. For example when a call to a remote service hangs
we can send back a partial response of other work that was completed rather
than just "hanging" or failing completely.
Logging
-- Built-in support to log the duration of all Minion tasks to make future analysis
+- Built-in support to log the duration of all minion tasks to make future analysis
of performance issues much easier
-- Logs any exceptions thrown to assist fwith problem diagnosis
+- Logs any exceptions thrown to assist with problem diagnosis
+- Logging tags from the current thread are propagated to the minions thread
+- The name of the thread in log entries is set to the description supplied for
+ the minion to make it easy to distinguish log entries by minion / thread
+Rails Support
+
+- When used in a Rails environment the current scope of specified models can be
+ propagated to the minions thread
+
## Example
Simple example
```ruby
-ParallelMinion::Minion.new(10.days.ago, description: 'Doing something else in parallel', timeout: 1000) do |date|
+minion = ParallelMinion::Minion.new(10.days.ago, description: 'Doing something else in parallel', timeout: 1000) do |date|
MyTable.where('created_at <= ?', date).count
end
+
+# Do other work here...
+
+# Retrieve the result of the minion
+count = minion.result
+
+puts "Found #{count} records"
```
## Example
-For example, in the code below there are several steps that are performed sequentially:
+For example, in the code below there are several steps that are performed
+sequentially and does not yet use minions:
```ruby
# Contrived example to show how to do parallel code execution
# with (unreal) sample durations in the comments
@@ -188,15 +206,41 @@
end
```
The above #process_request method should now take on average 1,810 milli-seconds
which is significantly faster than the 3,780 milli-seconds it took to perform
-the exact same request, but using only a single thread
+the exact same request prior to using minions.
-The exact breakdown of which calls to do in the main thread versus a Minion is determined
+The exact breakdown of which calls to do in the main thread versus a minion is determined
through experience and trial and error over time. The key is logging the duration
-of each call which Minion does by default so that the exact processing breakdown
+of each call which minion does by default so that the exact processing breakdown
can be fine-tuned over time.
+
+## Disabling Minions
+
+In the event that strange problems are occurring in production and no one is
+sure if it is due to running the minion tasks in parallel, it is simple to make
+all minion tasks run in the calling thread.
+
+It may also be useful to disable minions on a single production server to compare
+its performance to that of the servers running with minions active.
+
+To disable minions / make them run in the calling thread, add the following
+lines to config/environments/production.rb:
+
+```ruby
+ # Make minions run immediately in the current thread
+ config.parallel_minion.enabled = false
+```
+
+## Notes:
+
+- When using JRuby it is important to enable it's built-in thread-pooling by
+ adding the following line to .jrubyrc, or setting the appropriate command line option:
+
+```ruby
+thread.pool.enabled=true
+```
Meta
----
* Code: `git clone git://github.com/reidmorrison/parallel_minion.git`