README.md in fluent-plugin-aurora-slowquerylog-0.0.3 vs README.md in fluent-plugin-aurora-slowquerylog-0.0.4
- old
+ new
@@ -1,29 +1,67 @@
-# Fluent::Plugin::Aurora::Slowquerylog
+# [Fluentd](http://fluentd.org) input plugin for AWS RDS Aurora
-Welcome to your new gem! In this directory, you'll find the files you need to be able to package up your Ruby library into a gem. Put your Ruby code in the file `lib/fluent/plugin/aurora/slowquerylog`. To experiment with that code, run `bin/console` for an interactive prompt.
+## Overview
+This fluent input plugin collects RDS Aurora slowquery log with ParameterGroup option `log_output=FILE`.
+This plugin fetches only the difference between the latest and previous fetched slow query log.
-TODO: Delete this and the text above, and describe your gem
+## Background
+There are a lot of RDS mysql slowlog input plugins that collect mysql slowquery logs with ParameterGroup option `log_output=TABLE`.
+However if you use RDS Aurora with the option, there are two following problems:
-## Installation
+### 1. mysql.slow_log table is operated with Engine=CSV
-Add this line to your application's Gemfile:
+mysql.slow_log table is operated with Engine=CSV and we cannot add index to the table.
+So if tons of slow queries are registered to mysql.slow_log, your `SELECT * FROM mysql.slow_log WHERE condition` queries also become slow.
-```ruby
-gem 'fluent-plugin-aurora-slowquerylog'
+### 2. No way to rotate Aurora slowlog table
+
+mysql.slow_log table on RDS mysql can be purged by using `CALL mysql.rds_rotate_slow_log`. On the other hand, the stored procedure executed on Aurora makes following error.
+
```
+> CALL mysql.rds_rotate_slow_log;
+ERROR 1289 (HY000): The 'CSV' feature is disabled; you need MySQL built with 'CSV' to have it working
+```
-And then execute:
+According to the above reasons, I have implemented input plugin with `log_output=FILE` for Aurora.
- $ bundle
+## Installation
-Or install it yourself as:
+```ruby
+$ fluent-gem install fluent-plugin-aurora-slowquerylog --no-document
+```
- $ gem install fluent-plugin-aurora-slowquerylog
+## Input: How It Works
+TBW
## Usage
+### Sample configuration
-TODO: Write usage instructions here
+```
+<source>
+ @type aurora_slowquerylog
+ tag aurora.slowlog
+ db_instance_identifier aurora_node_id
+ region us-east-1
+ log_file_name slowquery/mysql-slowquery.log
+ aurora_state_file /var/run/fluentd/aurora_state
+ log_fetch_interval 30 #optionnal
+ aws_access_key_id your_aws_access_key_id #optionnal
+ aws_secret_access_key your_aws_secret_access_key #optionnal
+ filename_contains mysql-slowquery.log # default 'mysql-slowquery.log'
+</source>
+```
+
+* **tag** tag name of events
+* **db_instance_identifier** AWS Aurora node id
+* **region** AWS region name
+* **log_file_name** RDS slowlog name. Currently we cannot change file name from 'slowquery/mysql-slowquery.log'
+* **aurora_state_file** state file that keeps maker information and current & last slowquery log name
+* **log_fetch_interval** interval time(second) for log fetch (optional)
+* **aws_access_key_id** AWS access key id. For AWS user IAM instance profile is recommended without using this option (optional)
+* **aws_secret_access_key** AWS secret access key. For AWS user IAM instance profile is recommended without using this option (optional)
+* **filename_contains** filter condition for fetching slow query log (optional)
+
## Development
After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.