= MongoDB plugin for Fluent

== Component

=== MongoOutput

Store fluent-event as MongoDB Document to MongoDB database.

MongoOutput set 'time' field to a document by default.
You set _false_ to 'include_time_key' parameter if you disable this behaivor.

==== tag mapped mode

Tag mapped to MongoDB collection automatically.

=== MongoBackupOutput

Store fluent-event to local capped collection for backup.

=== MongoTailInput

Tail capped collection to input data.

== Configuration

=== MongoOutput

    <match mongo.**>
      type mongo
      database fluent
      collection test

      # Following attibutes are optional
      host fluenter
      port 10000

      # You can use 'capped' if you want to use capped collection
      capped
      capped_size 100m

      # Other buffer configurations here
    </match>

==== Tag mapped mode

Use 'tag_mapped' parameter.

If tag name is "foo.bar", auto create collection "foo.bar" and insert data.

    <match forward.*>
      type mongo
      database fluent

      # If You use 'tag_mapped', then tag mapped mode enabled.
      tag_mapped

      # If tag is "forward.foo.bar", then prefix "forward." is removed.
      # Collection name to insert is "foo.bar".
      remove_tag_prefix forward.

      # This configuration is used if tag not found. Default is 'untagged'.
      collection misc

      # Other configurations here
    </match>

==== NOTE

===== Broken data as a BSON

Fluentd event sometimes has an invalid record as a BSON.
In such case, mongo plugin marshals an invalid record using Marshal.dump
and re-inserts its to same collection.

Example:

    {"key1": "invalid value", "key2": "valid value", "time": ISODate("2012-01-15T21:09:53Z") }

to

    {"__broken_data": "Marshal.dump result", "time": ISODate("2012-01-15T21:09:53Z") }

mongo-ruby-driver cannot detect invalid attribute,
so mongo plugin marshals all attributes excluding Fluentd keys(tag_key and time_key).

If you ignores an invalid record, please set 'ignore_invalid_record true' in match.

===== Buffer size limitation

MongoDB's output plugins have the limitation of buffer size.
Because MongoDB and Ruby-Driver checks the total object size at each insertion.
If total object size gets over the size limitation,
MongoDB returns error or Ruby-Driver raises an Exception.

So, MongoDB's output plugins reset :buffer_chunk_limit if configurated value is larger than above limitation.
- Before v1.8, max of :buffer_chunk_limit is 2MB
- After v1.8, max of :buffer_chunk_limit is 10MB

=== MongoBackupOutput

Use mongo_backup type in match. mongo_backup alwalys use capped collection.

    <match ...>
      type mongo_backup
      capped_size 100m

      <store>
        type tcp
        host 192.168.0.13
        ...
      </store>
    </match>

=== Tail capped collection

Use mongo_tail type in source. 

    <source>
      type mongo_tail
      database fluent
      collection capped_log

      tag app.mongo_log

      # Convert 'time'(BSON's time) to fluent time(Unix time).
      time_key time

      # You can store last ObjectId to tail over server's shutdown
      id_store_file /Users/repeatedly/devel/fluent-plugin-mongo/last_id
    </source>

== TODO

=== More configuration

- Authentication
- Select insert or update
- etc

== Tool

You can tail mongo capped collection.

    mongo-tail -f

== Copyright

Copyright:: Copyright (c) 2011- Masahiro Nakagawa
License::   Apache License, Version 2.0