README.md in fluent-plugin-groupcounter-0.1.0 vs README.md in fluent-plugin-groupcounter-0.2.0
- old
+ new
@@ -1,43 +1,124 @@
# fluent-plugin-groupcounter
-## Component
+Fluentd plugin to count like SELECT COUNT(\*) GROUP BY.
-### GroupCounterOutput
-
-Fluentd plugin to count like COUNT(\*) GROUP BY
-
## Configuration
-## GroupCounterOutput
+Assume inputs are coming as followings:
- <source>
- type tail
- path /var/log/httpd-access.log
- tag apache.access
- format apache
- </source>
+ apache.access: {"code":"200", "method":"GET", "path":"/index.html", "reqtime":"1.001" }
+ apache.access: {"code":"404", "method":"GET", "path":"/foo.html", "reqtime":"2.002" }
+ apache.access: {"code":"200", "method":"GET", "path":"/index.html", "reqtime":"3.003" }
+Think of quering `SELECT COUNT(\*) GROUP BY code,method,path`. Configuration becomes as below:
+
<match apache.access>
type groupcounter
- count_interval 5s
aggregate tag
output_per_tag true
tag_prefix groupcounter
group_by_keys code,method,path
</match>
-Output like below
+Output becomes like
- groupcounter.apache.access: {"200_GET_/index.html_count":1,"200_GET_/index.html_rate":0.2,"200_GET_/index.html_percentage":100.0}
+ groupcounter.apache.access: {"200_GET_/index.html_count":2, "404_GET_/foo.html_count":1}
-## TODO
+## Parameters
-* tests
-* documents
+* group\_by\_keys (semi-required)
+ Specify keys in the event record for grouping. `group_by_keys` or `group_by_expression` is required.
+
+* delimiter
+
+ Specify the delimiter to join `group_by_keys`. Default is '_'.
+
+* group\_by\_expression (semi-required)
+
+ Use an expression to group the event record. `group_by_keys` or `group_by_expression` is required.
+
+ For examples, for the exampled input above, the configuration as below
+
+ group_by_expression ${method}${path}/${code}
+
+ gives you an output like
+
+ groupcounter.apache.access: {"GET/index.html/200_count":1, "GET/foo.html/400_count":1}
+
+ SECRET TRICK: You can write a ruby code in the ${} placeholder like
+
+ group_by_expression ${method}${path.split(".")[0]}/${code[0]}xx
+
+ This gives an output like
+
+ groupcounter.apache.access: {"GET/index/2xx_count":1, "GET/foo/4xx_count":1}
+
+* tag
+
+ The output tag. Default is `groupcount`.
+
+* tag\_prefix
+
+ The prefix string which will be added to the input tag. `output_per_tag yes` must be specified together.
+
+* input\_tag\_remove\_prefix
+
+ The prefix string which will be removed from the input tag.
+
+* count\_interval
+
+ The interval time to count in seconds. Default is `60`.
+
+* unit
+
+ The interval time to monitor specified an unit (either of `minute`, `hour`, or `day`).
+ Use either of `count_interval` or `unit`.
+
+* store\_file
+
+ Store internal data into a file of the given path on shutdown, and load on starting.
+
+* max\_key
+
+ Specify key name in the event record to do `SELECT COUNT(\*),MAX(key_name) GROUP BY`.
+
+ For examples, for the exampled input above, adding the configuration as below
+
+ max_key reqtime
+
+ gives you an output like
+
+ groupcounter.apache.access: {"200_GET_/index.html_reqtime_max":3.003, "404_GET_/foo.html_reqtime_max":2.002}
+
+* min\_key
+
+ Specify key name in the event record to do `SELECT COUNT(\*),MIN(key_name) GROUP BY`.
+
+* avg\_key
+
+ Specify key name in the event record to do `SELECT COUNT(\*),AVG(key_name) GROUP BY`.
+
+* count\_suffix
+
+ Default is `_count`
+
+* max\_suffix
+
+ Default is `_max`. Should be used with `max_key` option.
+
+* min\_suffix
+
+ Default is `_min`. Should be used with `min_key` option.
+
+* avg\_suffix
+
+ Default is `_avg`. Should be used with `avg_key` option.
+
## Copyright
* Copyright
* Copyright (c) 2012- Ryosuke IWANAGA (riywo)
+ * Copyright (c) 2013- Naotoshi SEO (sonots)
* License
* Apache License, Version 2.0