README in forkoff-0.0.4 vs README in forkoff-1.1.0
- old
+ new
@@ -7,10 +7,11 @@
brain-dead simple parallel processing for ruby
URI
http://rubyforge.org/projects/codeforpeople
+ http://github.com/ahoward/forkoff
INSTALL
gem install forkoff
@@ -18,52 +19,13 @@
forkoff works for any enumerable object, iterating a code block to run in a
child process and collecting the results. forkoff can limit the number of
child processes which is, by default, 2.
-HISTORY
- 0.0.4
- - code re-org
- - add :strategy option
- - default number of processes is 2, not 8
-
- 0.0.1
-
- - updated to use producer threds pushing onto a SizedQueue for each consumer
- channel. in this way the producers do not build up a massize parllel data
- structure but provide data to the consumers only as fast as they can fork
- and proccess it. basically for a 4 process run you'll end up with 4
- channels of size 1 between 4 produces and 4 consumers, each consumer is a
- thread popping of jobs, forking, and yielding results.
-
- - removed use of Queue for capturing the output. now it's simply an array
- of arrays which removed some sync overhead.
-
- - you can configure the number of processes globally with
-
- Forkoff.default['proccess'] = 4
-
- - you can now pass either an options hash
-
- forkoff( :processes => 2 ) ...
-
- or plain vanilla number
-
- forkoff( 2 ) ...
-
- to the forkoff call
-
- - default number of processes is 8, not 2
-
-
- 0.0.0
-
- initial version
-
-
SAMPLES
+
<========< samples/a.rb >========>
~ > cat samples/a.rb
# forkoff makes it trivial to do parallel processing with ruby, the following
@@ -74,12 +36,12 @@
%w( hey you ).forkoff!{|word| puts "#{ word } from #{ Process.pid }"}
~ > ruby samples/a.rb
- hey from 1032
- you from 1033
+ hey from 7907
+ you from 7908
<========< samples/b.rb >========>
~ > cat samples/b.rb
@@ -105,11 +67,11 @@
puts "elapsed: #{ elapsed }"
puts "results: #{ results.inspect }"
~ > ruby samples/b.rb
- elapsed: 4.25545883178711
+ elapsed: 4.19184589385986
results: [0, 1, 4, 9, 16, 25, 36, 49]
<========< samples/c.rb >========>
@@ -149,75 +111,74 @@
end
~ > ruby samples/c.rb
- pid: 1048
- elapsed: 3.14415812492371
+ pid: 7922
+ elapsed: 3.37899208068848
---
a: |
- -+- 01048 ahoward ruby -Ilib samples/c.rb
- |-+- 01049 ahoward ruby -Ilib samples/c.rb
- |-+- 01050 ahoward ruby -Ilib samples/c.rb
- \-+- 01051 ahoward ruby -Ilib samples/c.rb
+ -+- 07922 ahoward ruby -Ilib samples/c.rb
+ |-+- 07923 ahoward ruby -Ilib samples/c.rb
+ |-+- 07924 ahoward (ruby)
+ \-+- 07925 ahoward ruby -Ilib samples/c.rb
---
b: |
- -+- 01048 ahoward ruby -Ilib samples/c.rb
- |-+- 01049 ahoward (ruby)
- |-+- 01050 ahoward ruby -Ilib samples/c.rb
- \-+- 01051 ahoward ruby -Ilib samples/c.rb
+ -+- 07922 ahoward ruby -Ilib samples/c.rb
+ |-+- 07923 ahoward ruby -Ilib samples/c.rb
+ |-+- 07924 ahoward ruby -Ilib samples/c.rb
+ \-+- 07925 ahoward ruby -Ilib samples/c.rb
---
c: |
- -+- 01048 ahoward ruby -Ilib samples/c.rb
- |-+- 01049 ahoward ruby -Ilib samples/c.rb
- |-+- 01050 ahoward ruby -Ilib samples/c.rb
- \-+- 01051 ahoward ruby -Ilib samples/c.rb
+ -+- 07922 ahoward ruby -Ilib samples/c.rb
+ |-+- 07923 ahoward ruby -Ilib samples/c.rb
+ |-+- 07924 ahoward (ruby)
+ \-+- 07925 ahoward ruby -Ilib samples/c.rb
---
d: |
- -+- 01048 ahoward ruby -Ilib samples/c.rb
- |-+- 01061 ahoward ruby -Ilib samples/c.rb
- |-+- 01062 ahoward ruby -Ilib samples/c.rb
- \-+- 01063 ahoward ruby -Ilib samples/c.rb
+ -+- 07922 ahoward ruby -Ilib samples/c.rb
+ |-+- 07932 ahoward ruby -Ilib samples/c.rb
+ |--- 07933 ahoward ruby -Ilib samples/c.rb
+ \--- 07934 ahoward ruby -Ilib samples/c.rb
---
e: |
- -+- 01048 ahoward ruby -Ilib samples/c.rb
- |-+- 01061 ahoward (ruby)
- |-+- 01062 ahoward ruby -Ilib samples/c.rb
- \-+- 01063 ahoward ruby -Ilib samples/c.rb
+ -+- 07922 ahoward ruby -Ilib samples/c.rb
+ |--- 07932 ahoward (ruby)
+ |-+- 07933 ahoward ruby -Ilib samples/c.rb
+ \-+- 07934 ahoward (ruby)
---
f: |
- -+- 01048 ahoward ruby -Ilib samples/c.rb
- |-+- 01061 ahoward ruby -Ilib samples/c.rb
- |-+- 01062 ahoward ruby -Ilib samples/c.rb
- \-+- 01063 ahoward ruby -Ilib samples/c.rb
+ -+- 07922 ahoward ruby -Ilib samples/c.rb
+ |--- 07932 ahoward (ruby)
+ |-+- 07933 ahoward ruby -Ilib samples/c.rb
+ \-+- 07934 ahoward ruby -Ilib samples/c.rb
---
g: |
- -+- 01048 ahoward ruby -Ilib samples/c.rb
- |-+- 01090 ahoward ruby -Ilib samples/c.rb
- |-+- 01091 ahoward ruby -Ilib samples/c.rb
- \-+- 01092 ahoward ruby -Ilib samples/c.rb
+ -+- 07922 ahoward ruby -Ilib samples/c.rb
+ |-+- 07941 ahoward ruby -Ilib samples/c.rb
+ |--- 07942 ahoward ruby -Ilib samples/c.rb
+ \--- 07943 ahoward ruby -Ilib samples/c.rb
---
h: |
- -+- 01048 ahoward ruby -Ilib samples/c.rb
- |-+- 01090 ahoward ruby -Ilib samples/c.rb
- |-+- 01091 ahoward ruby -Ilib samples/c.rb
- \-+- 01092 ahoward ruby -Ilib samples/c.rb
+ -+- 07922 ahoward ruby -Ilib samples/c.rb
+ |-+- 07941 ahoward (ruby)
+ |-+- 07942 ahoward ruby -Ilib samples/c.rb
+ \--- 07943 ahoward ruby -Ilib samples/c.rb
---
i: |
- -+- 01048 ahoward ruby -Ilib samples/c.rb
- |-+- 01090 ahoward ruby -Ilib samples/c.rb
- |-+- 01091 ahoward ruby -Ilib samples/c.rb
- \-+- 01092 ahoward ruby -Ilib samples/c.rb
+ -+- 07922 ahoward ruby -Ilib samples/c.rb
+ |--- 07942 ahoward (ruby)
+ \-+- 07943 ahoward ruby -Ilib samples/c.rb
<========< samples/d.rb >========>
@@ -234,9 +195,60 @@
puts "#{ word } from #{ Process.pid }"
end
~ > ruby samples/d.rb
- hey from 1102
- you from 1103
- guys from 1104
+ hey from 7953
+ you from 7954
+ guys from 7955
+
+
+HISTORY
+ 1.1.0
+ - move to a model with one work queue and signals sent from consumers to
+ producer to noitify ready state. this let's smaller jobs race through a
+ single process even while a larger job may have one sub-process bound up.
+ incorporates a fix from http://github.com/fredrikj/forkoff which meant
+ some processes would lag behind when jobs didn't have similar execution
+ times.
+
+ 1.0.0
+ - move to github
+
+ 0.0.4
+ - code re-org
+ - add :strategy option
+ - default number of processes is 2, not 8
+
+ 0.0.1
+
+ - updated to use producer threds pushing onto a SizedQueue for each consumer
+ channel. in this way the producers do not build up a massize parllel data
+ structure but provide data to the consumers only as fast as they can fork
+ and proccess it. basically for a 4 process run you'll end up with 4
+ channels of size 1 between 4 produces and 4 consumers, each consumer is a
+ thread popping of jobs, forking, and yielding results.
+
+ - removed use of Queue for capturing the output. now it's simply an array
+ of arrays which removed some sync overhead.
+
+ - you can configure the number of processes globally with
+
+ Forkoff.default['proccess'] = 4
+
+ - you can now pass either an options hash
+
+ forkoff( :processes => 2 ) ...
+
+ or plain vanilla number
+
+ forkoff( 2 ) ...
+
+ to the forkoff call
+
+ - default number of processes is 8, not 2
+
+
+ 0.0.0
+
+ initial version