README.md in barnyard_harvester-0.0.8 vs README.md in barnyard_harvester-0.0.9
- old
+ new
@@ -1,8 +1,10 @@
# BarnyardHarvester
The Harvester gem provides a simple interface where you can iterate your data source and send records to the sync engine. The default backend storage is Redis. Since the workers will use Resque which requires Redis, this make sense to use Redis to cache the data. However, any backend cache can be implemented.
+I've now implemented using DynamoDB for the backend and SQS for the messages.
+
Your data sources are called "crops". You must assign a unique integer 1..100 to each crop. This is the integer that is used to create the Redis collection. By default Redis only allows a maximum of 16 databases, so this must be changed if you go above 16.
WARNING! Do not use crop number 1-9 in production, they are reserved for system testing.
The sync engine keeps a copy of your data source in a Redis databased indexed on the data source's primary key. When you sync your data, the engine uses this "cached" copy of the data to determine adds, deletes and changes. None of the data is inspected and the sync engine does not care what is in the data as long as it can be marshaled into a JSON string. The sync engine automatically does the conversion to and from JSON, so all you pass in is an object that responds to .to_json.
\ No newline at end of file