README.md in fluent-plugin-querycombiner-0.0.1 vs README.md in fluent-plugin-querycombiner-0.0.2

- old
+ new

@@ -76,11 +76,11 @@ } ``` ### Replace some field names -If messages has the same fields, these are overwritten in the combination process. You can use `replace` sentence in `<catch>` and `<dump>` blocks to avoid overwrite such fields. +If messages has the same fields, these are overwritten in the combination process. You can use `replace` sentence in `<catch>` and `<dump>` blocks to avoid overwriting such fields. For example, you have some event messages like: ``` { @@ -111,11 +111,10 @@ buffer_size 1000 # max queries to store in redis <catch> condition status == 'event-start' replace time => time_start - </catch> <dump> condition status == 'event-finish' replace time => time_finish @@ -133,10 +132,19 @@ "time_start": "2001-02-03T04:05:06Z", "time_finish": "2001-02-03T04:15:11Z" } ``` +You can also replace multiple fields joined by comma(`,`): + +``` +<catch> + condition status == 'event-start' + replace time => time_start, condition => condition_start +</catch> +``` + ### \<release\> block In previous examples, messages with `"status": "event-start"` will be watched by plugin immediately. Suppose some error events occur and you don't want to watch or combine these messages. @@ -164,11 +172,11 @@ You cannot use `replace` sentence in the `<release>` block. ### \<prolong\> block -Suppose your `query_ttl` is **600** (10 minutes) and almost events are finished within **10 minutes**. But occasionally very-long events occur which finish about **1 hours**. These very-long events send `status: 'event-continue'` messages every 5 minutes for keep-alive. +Suppose your `query_ttl` is **600** (10 minutes) and almost events are finished within **10 minutes**. But occasionally very-long events occur which finish about **1 hour**. These very-long events send `status: 'event-continue'` messages every 5 minutes for keep-alive. In this case you can use `<prolong>` block to reset expired time. ``` <prolong> @@ -178,11 +186,39 @@ You cannot use `replace` sentence in the `<prolong>` block. Also you cannot combine messages which defined `<prolong>` blocks. +### Record time of the event +If you combine events, time of the events will be lost except defined in `<dump>` block. + +If you want record time of the event, you can define `time` sentence in `<catch>` and `<dump>` blocks. + +For example, if you configure your fluentd configuration like below, + +``` + <catch> + condition status == 'event-start' + replace time => time_start, condition => condition_start + time time-catch + </catch> +``` + +you can record time in `time-catch` field in the result. + +``` +{ + "event_id": "01234567", + "status": "event-finish", + "time-catch": 1414715801.112015, +} +``` + +You can set time formats by `time_format` configuration. + + ## Configuration ### tag The tag prefix for emitted event messages. By default it's `query_combiner`. @@ -191,33 +227,40 @@ ### redis_retry How many times should the plugin retry when performing a redis operation before raising a error. By default it's 3. -### querl_ttl +### query_ttl The inactive expire time in seconds. By default it's **1800** (30 minutes). ### buffer_size The max queries to store in redis. By default it's **1000**. ### remove_interval The interval time to delete expired or overflowed queries which configured by `query_ttl` and `buffer_size`. By default it's 10 [sec]. - ### redis_key_prefix The key prefix for data stored in Redis. By default it's `query_combiner:`. ### query_identify Indicates how to extract the query identity from event record. It can be set as a single field name or multiple field names join by comma (`,`). +### time_format +The time format for recording time of the events. Default is `$time` which holds event time. You can also use [Ruby's Time module](http://www.ruby-doc.org/core/Time.html). +If you want write ISO8601 format (e.g. `2014-10-31T09:32:57+09:00`), you can configure like below. + +``` +time_format Time.at($time).iso8601 +``` + + ## TODO - Multi-query combination -- Support hyphen `-` and dollar `$` contained field names ## Copyright Copyright:: Copyright (c) 2014- Takahiro Kamatani