README.rdoc in seamusabshere-data_miner-0.2.0 vs README.rdoc in seamusabshere-data_miner-0.2.1
- old
+ new
@@ -6,28 +6,12 @@
Put this in <tt>config/environment.rb</tt>:
config.gem 'seamusabshere-data_miner', :lib => 'data_miner', :source => 'http://gems.github.com'
-Put this in <tt>lib/tasks/data_miner_tasks.rake</tt>: (unfortunately I don't know a way to automatically include gem tasks, so you have to do this manually for now)
+You need to define <tt>mine_data</tt> blocks in your ActiveRecord models. For example, in <tt>app/models/country.rb</tt>:
- namespace :data_miner do
- task :mine => :environment do
- DataMiner.mine :class_names => ENV['CLASSES'].to_s.split(/\s*,\s*/).flatten.compact
- end
- end
-
-You need to specify what order to mine data. For example, in <tt>config/initializers/data_miner_config.rb</tt>:
-
- DataMiner.enqueue do |queue|
- queue << Country # class whose data should be mined 1st
- queue << Airport # class whose data should be mined 2nd
- # etc
- end
-
-You need to define <tt>mine_data</tt> blocks. For example, in <tt>app/models/country.rb</tt>:
-
class Country < ActiveRecord::Base
mine_data do |step|
# import country names and country codes
step.import :url => 'http://www.cs.princeton.edu/introcs/data/iso3166.csv' do |attr|
attr.key :iso_3166, :name_in_source => 'country code'
@@ -35,11 +19,11 @@
attr.store :name, :name_in_source => 'country'
end
end
end
-To complete the example, in <tt>app/models/airport.rb</tt>:
+...and in <tt>app/models/airport.rb</tt>:
class Airport < ActiveRecord::Base
belongs_to :country
mine_data do |step|
@@ -52,9 +36,29 @@
attr.store :iata_code, :field_number => 3
attr.store :latitude, :field_number => 5
attr.store :longitude, :field_number => 6
end
end
+ end
+
+Put this in <tt>lib/tasks/data_miner_tasks.rake</tt>: (unfortunately I don't know a way to automatically include gem tasks, so you have to do this manually for now)
+
+ namespace :data_miner do
+ task :mine => :environment do
+ DataMiner.mine :class_names => ENV['CLASSES'].to_s.split(/\s*,\s*/).flatten.compact
+ end
+
+ task :map_to_attrs => :environment do
+ DataMiner.map_to_attrs ENV['METHOD'], :class_names => ENV['CLASSES'].to_s.split(/\s*,\s*/).flatten.compact
+ end
+ end
+
+You need to specify what order to mine data. For example, in <tt>config/initializers/data_miner_config.rb</tt>:
+
+ DataMiner.enqueue do |queue|
+ queue << Country # class whose data should be mined 1st
+ queue << Airport # class whose data should be mined 2nd
+ # etc
end
Once you have (1) set up the order of data mining and (2) defined <tt>mine_data</tt> blocks in your classes, you can:
$ rake data_miner:mine