README.rdoc in harvestdor-indexer-0.0.13 vs README.rdoc in harvestdor-indexer-1.0.0
- old
+ new
@@ -1,12 +1,11 @@
= Harvestdor::Indexer
{<img src="https://travis-ci.org/sul-dlss/harvestdor-indexer.svg" alt="Build Status" />}[https://travis-ci.org/sul-dlss/harvestdor-indexer]
-{<img src="https://coveralls.io/repos/sul-dlss/harvestdor-indexer/badge.png" alt="Coverage Status" />}[https://coveralls.io/r/sul-dlss/harvestdor-indexer]{<img src="https://gemnasium.com/sul-dlss/harvestdor-indexer.svg" alt="Dependency Status" />}[https://gemnasium.com/sul-dlss/harvestdor-indexer]{<img src="https://badge.fury.io/rb/harvestdor-indexer.svg" alt="Gem Version" />}[http://badge.fury.io/rb/harvestdor-indexer]
+{<img src="https://coveralls.io/repos/sul-dlss/harvestdor-indexer/badge.png" alt="Coverage Status" />}[https://coveralls.io/r/sul-dlss/harvestdor-indexer]
+{<img src="https://gemnasium.com/sul-dlss/harvestdor-indexer.svg" alt="Dependency Status" />}[https://gemnasium.com/sul-dlss/harvestdor-indexer]
+{<img src="https://badge.fury.io/rb/harvestdor-indexer.svg" alt="Gem Version" />}[http://badge.fury.io/rb/harvestdor-indexer]
-
-
-
A Gem to harvest meta/data from DOR and the skeleton code to index it and write to Solr.
== Installation
Add this line to your application's Gemfile:
@@ -40,18 +39,15 @@
open_timeout: 180
See spec/config/ap.yml for an example.
You will want to copy that file and change the following settings:
1. log_name
-2. default_set (in OAI harvesting params section)
-3. other OAI harvesting params
-4. blacklist or whitelist if you are using them
+2. default_set
+3. blacklist or whitelist if you are using them
-You can also pass in non-default configurations as a hash
+Update the dor-fetcher-client.yml file in the config directory with the location of the URL of the dor-fetcher-service provider. The defaulted value is the 3000 port for a localhost - dor_fetcher_service_url: http://127.0.0.1:3000
- indexer = Harvestdor::Indexer.new({:oai_repository_url => 'http://my_oai.org, :default_from_date => '2012-12-01'})
-
=== Override the Harvestdor::Indexer.index method
In your code, override this method from the Harvestdor::Indexer class
# create Solr doc for the druid and add it to Solr, unless it is on the blacklist.
@@ -92,21 +88,25 @@
require 'bundler/setup'
require 'your_indexer'
end
config_yml_path = ARGV.pop
if config_yml_path.nil?
- puts "** You must provide the full path to a config yml file **"
+ puts "** You must provide the full path to a collection config yml file **"
exit
end
- indexer = Harvestdor::Indexer.new(config_yml_path, opts)
+ if client_config_path.nil?
+ puts "** You must provide the full path to dor-fetcher-client config yml file **"
+ exit
+ end
+ indexer = Harvestdor::Indexer.new(config_yml_path, client_config_path, opts)
indexer.harvest_and_index
Then you run the script like so:
./bin/indexer config/(your coll).yml
-I suggest you run your code on harvestdor-dev, as it is already set up to be able to harvest from the DOR OAI provider
+I suggest you run your code on harvestdor-dev, as it is already set up to be able to harvest from the DorFetcher
== Contributing
# Fork it
@@ -116,9 +116,10 @@
# Push to the branch (`git push origin my-new-feature`)
# Create new Pull Request
== Releases
+* <b>1.0.0</b> Replaced OAI harvesting mechanism with dor-fetcher
* <b>0.0.13</b> Upgrade to latest faraday HTTP client syntax; Use retries gem (https://github.com/ooyala/retries) to make retrying of index process more robust
* <b>0.0.12</b> fix total_object nil error
* <b>0.0.11</b> fix error_count and success_count, allow setting of max-tries (retry solr add if error)
* <b>0.0.7</b> adding additional logging of error, success counts, and time to index and harvest
* <b>0.0.6</b> tweak error handling for public xml pieces