README.md in fluent-plugin-arrow-0.0.1 vs README.md in fluent-plugin-arrow-0.0.2
- old
+ new
@@ -1,11 +1,16 @@
# fluent-plugin-arrow
-[Fluentd](https://fluentd.org/) formatter plugin to do something.
+[Fluentd](https://fluentd.org/) buffer plugin to output Apache Arrow and Parquet format.
-TODO: write description for you plugin.
+## Prerequisite
+- [Apache Arrow c++](https://github.com/apache/arrow/tree/master/cpp) (with -DARROW_PARQUET=ON)
+- [Apache Arrow c_glib](https://github.com/apache/arrow/tree/master/c_glib)
+- [red-arrow](https://github.com/apache/arrow/tree/master/ruby/red-arrow)
+- [red-parquet](https://github.com/apache/arrow/tree/master/ruby/red-parquet)
+
## Installation
### RubyGems
```
@@ -29,10 +34,34 @@
## Configuration
You can generate configuration template:
```
-$ fluent-plugin-config-format formatter arrow
+<match arrow>
+ @type file
+
+ path arrow_test
+
+ <buffer>
+ @type arrow_memory
+ arrow_format arrow # or parquet
+
+ schema [
+ {"name": "key1", "type": "string"},
+ {"name": "key2", "type": "uint64"},
+ {"name": "key3", "type": "timestamp", "unit": "milli"},
+ {"name": "key4", "type": "list", "value_type": {"name": "value", "type": "uint64"}},
+ {"name": "key5", "type": "struct", "fields": [
+ {"name": "bar1", "type": "uint64"},
+ {"name": "bar2", "type": "list", "value_type": {"name": "value", "type": "string"}}
+ ]}
+ ]
+ </buffer>
+
+ <format>
+ @type arrow
+ </format>
+</match>
```
You can copy and paste generated documents here.
## Copyright