README.rdoc in data_miner-0.2.6 vs README.rdoc in data_miner-0.3.0
- old
+ new
@@ -6,29 +6,29 @@
Put this in <tt>config/environment.rb</tt>:
config.gem 'data_miner'
-You need to define <tt>mine_data</tt> blocks in your ActiveRecord models. For example, in <tt>app/models/country.rb</tt>:
+You need to define <tt>data_miner</tt> blocks in your ActiveRecord models. For example, in <tt>app/models/country.rb</tt>:
class Country < ActiveRecord::Base
- mine_data do |step|
+ data_miner do |step|
# import country names and country codes
step.import :url => 'http://www.cs.princeton.edu/introcs/data/iso3166.csv' do |attr|
- attr.key :iso_3166, :name_in_source => 'country code'
- attr.store :iso_3166, :name_in_source => 'country code'
- attr.store :name, :name_in_source => 'country'
+ attr.key :iso_3166, :field_name => 'country code'
+ attr.store :iso_3166, :field_name => 'country code'
+ attr.store :name, :field_name => 'country'
end
end
end
...and in <tt>app/models/airport.rb</tt>:
class Airport < ActiveRecord::Base
belongs_to :country
- mine_data do |step|
+ data_miner do |step|
# import airport iata_code, name, etc.
step.import(:url => 'http://openflights.svn.sourceforge.net/viewvc/openflights/openflights/data/airports.dat', :headers => false) do |attr|
attr.key :iata_code, :field_number => 3
attr.store :name, :field_number => 0
attr.store :city, :field_number => 1
@@ -41,30 +41,26 @@
end
Put this in <tt>lib/tasks/data_miner_tasks.rake</tt>: (unfortunately I don't know a way to automatically include gem tasks, so you have to do this manually for now)
namespace :data_miner do
- task :mine => :environment do
- DataMiner.mine :class_names => ENV['CLASSES'].to_s.split(/\s*,\s*/).flatten.compact
+ task :run => :environment do
+ DataMiner.run :class_names => ENV['CLASSES'].to_s.split(/\s*,\s*/).flatten.compact
end
-
- task :map_to_attrs => :environment do
- DataMiner.map_to_attrs ENV['METHOD'], :class_names => ENV['CLASSES'].to_s.split(/\s*,\s*/).flatten.compact
- end
end
You need to specify what order to mine data. For example, in <tt>config/initializers/data_miner_config.rb</tt>:
DataMiner.enqueue do |queue|
queue << Country # class whose data should be mined 1st
queue << Airport # class whose data should be mined 2nd
# etc
end
-Once you have (1) set up the order of data mining and (2) defined <tt>mine_data</tt> blocks in your classes, you can:
+Once you have (1) set up the order of data mining and (2) defined <tt>data_miner</tt> blocks in your classes, you can:
- $ rake data_miner:mine
+ $ rake data_miner:run
==Complete example
~ $ rails testapp
~ $ cd testapp/
@@ -73,10 +69,10 @@
~/testapp $ rake db:migrate
~/testapp $ touch lib/tasks/data_miner_tasks.rb
[...edit per quick start...]
~/testapp $ touch config/initializers/data_miner_config.rake
[...edit per quick start...]
- ~/testapp $ rake data_miner:mine
+ ~/testapp $ rake data_miner:run
Now you should have
~/testapp $ ./script/console
Loading development environment (Rails 2.3.3)