README.rdoc in fluent-plugin-mongo-0.4.0 vs README.rdoc in fluent-plugin-mongo-0.5.0

- old
+ new

@@ -1,44 +1,63 @@ -= MongoDB output plugin for Fluent += MongoDB plugin for Fluent == Component === MongoOutput Store fluent-event as MongoDB Document to MongoDB database. MongoOutput set 'time' field to a document by default. You set -false- to 'include_time_key' parameter if you disable this behaivor. -=== MongoBackup +=== MongoBackupOutput -Store fluent-event to capped collection for backup. +Store fluent-event to local capped collection for backup. -== Configuratin +=== MongoTagOutputCollection +Tag mapping to MongoDB collection automatically. + +=== MongoTailInput + +Tail capped collection to input data. + +== Configuration + === MongoOutput <match mongo.**> type mongo database fluent collection test - # following attibutes are optional + # Following attibutes are optional host fluenter port 10000 # You can use 'capped' if you want to use capped collection - capped + capped true capped_size 100m # Other buffer configurations here </match> -=== Backup to local capped collection +==== NOTE -Use mongo_backup type. mongo_backup alwalys use capped collection. +MongoDB's output plugins have the limitation of buffer size. +Because MongoDB and Ruby-Driver checks the total object size at each insertion. +If total object size gets over the size limitation, +MongoDB returns error or Ruby-Driver raises an Exception. +So, MongoDB's output plugins reset :buffer_chunk_limit if configurated value is larger than above limitation. +- Before v1.8, max of :buffer_chunk_limit is 2MB +- After v1.8, max of :buffer_chunk_limit is 10MB + +=== MongoBackupOutput + +Use mongo_backup type in match. mongo_backup alwalys use capped collection. + <match ...> type mongo_backup capped_size 100m <store> @@ -46,21 +65,52 @@ host 192.168.0.13 ... </store> </match> +=== MongoTagOutputCollection + +Use mongo_tag_collection type in match. + +If tag name is "foo.bar", auto create collection "foo.bar" and insert data. + + <match forward.*> + type mongo_tag_collection + database fluent + + # This configuration is used if tag not found. Default is 'untagged' + collection misc + + # Tag "forward.foo.bar" remove prefix "forward.". + # Collection Mapping name is "foo.bar". + remove_prefix_collection forward. + </match> + +=== Tail capped collection + +Use mongo_tail type in source. + + <source> + type mongo_tail + database fluent + collection capped_log + + tag app.mongo_log + + # Convert 'time'(BSON's time) to fluent time(Unix time). + time_key time + + # You can store last ObjectId to tail over server's shutdown + id_store_file /Users/repeatedly/devel/fluent-plugin-mongo/last_id + </source> + == TODO === More configuration - Create Index - Select insert or update - etc - -=== Infer collection name - -Fluent tag is similar to database.collection in Mongo. -This feature makes configuration more easily. == Tool You can tail mongo capped collection.