README.md in kraps-0.8.0 vs README.md in kraps-0.9.0
- old
+ new
@@ -36,20 +36,25 @@
)
```
Afterwards, create a job class, which tells Kraps what your job should do.
Therefore, you create some class with a `call` method, and optionally some
-arguments. Let's create a simple job, which reads search log files to analyze
-how often search queries have been searched:
+arguments passed to its initializer. Let's create a simple job, which reads
+search log files to analyze how often search queries have been searched:
```ruby
class SearchLogCounter
- def call(start_date:, end_date:)
+ def initialize(start_date:, end_date:)
+ @start_date = start_date
+ @end_date = end_date
+ end
+
+ def call
job = Kraps::Job.new(worker: MyKrapsWorker)
job = job.parallelize(partitions: 128) do |collector|
- (Date.parse(start_date)..Date.parse(end_date)).each do |date|
+ (Date.parse(@start_date)..Date.parse(@end_date)).each do |date|
collector.call(date.to_s)
end
end
job = job.map do |date, _, collector|
@@ -212,10 +217,14 @@
collector.call(item)
end
end
```
+Please note, that `parallelize` itself is not parallelized but rather
+parallelizes the data you feed into Kraps within `parallelize` by splitting it
+into the number of `partitions` specified.
+
The block must use the collector to feed Kraps with individual items. The
items are used as keys and the values are set to `nil`.
* `map`: Maps the key value pairs to other key value pairs
@@ -343,14 +352,19 @@
each other. Let's assume that we additionally want to calculate a total number
of searches made:
```ruby
class SearchLogCounter
- def call(start_date:, end_date:)
+ def initialize(start_date:, end_date:)
+ @start_date = start_date
+ @end_date = end_date
+ end
+
+ def call
count_job = Kraps::Job.new(worker: SomeBackgroundWorker)
count_job = count_job.parallelize(partitions: 128) do |collector|
- (Date.parse(start_date)..Date.parse(end_date)).each do |date|
+ (Date.parse(@start_date)..Date.parse(@end_date)).each do |date|
collector.call(date.to_s)
end
end
count_job = count_job.map do |date, _, collector|