README.md in fluent-plugin-gcloud-pubsub-custom-1.0.2 vs README.md in fluent-plugin-gcloud-pubsub-custom-1.0.3
- old
+ new
@@ -17,10 +17,14 @@
- Create a project on Google Developer Console
- Add a topic of Cloud Pub/Sub to the project
- Add a pull style subscription to the topic
- Download your credential (json) or [set scope on GCE instance](https://cloud.google.com/compute/docs/api/how-tos/authorization)
+When using output plugin, you need to grant Pub/Sub Publisher and Pub/Sub Viewer role to IAM.
+
+Also, when using input plugin, you need to grant Pub/Sub Subscriber and Pub/Sub Viewer role to IAM.
+
## Requirements
| fluent-plugin-gcloud-pubsub-custom | fluentd | ruby |
|------------------------|---------|------|
| >= 1.0.0 | >= v0.14.0 | >= 2.1 |
@@ -54,15 +58,17 @@
topic <YOUR TOPIC>
autocreate_topic false
max_messages 1000
max_total_size 9800000
max_message_size 4000000
- buffer_type file
- buffer_path /path/to/your/buffer
- flush_interval 1s
- try_flush_interval 0.1
- format json
+ <buffer>
+ @type memory
+ flush_interval 1s
+ </buffer>
+ <format>
+ @type json
+ </format>
</match>
```
- `project` (optional)
- Set your GCP project.
@@ -82,14 +88,10 @@
- `max_total_size` (optional, default: `9800000` = `9.8MB`)
- Publishing messages bytesize per request to Cloud Pub/Sub. This parameter affects only message size. You should specify a little smaller value than quota.
- See https://cloud.google.com/pubsub/quotas#other_limits
- `max_message_size` (optional, default: `4000000` = `4MB`)
- Messages exceeding `max_message_size` are not published because Pub/Sub clients cannot receive it.
-- `buffer_type`, `buffer_path`, `flush_interval`, `try_flush_interval`
- - These are fluentd buffer configuration. See http://docs.fluentd.org/articles/buffer-plugin-overview
-- `format` (optional, default: `json`)
- - Set output format. See http://docs.fluentd.org/articles/out_file#format
### Pull messages
Use `gcloud_pubsub` input plugin.
@@ -103,15 +105,17 @@
subscription <YOUR SUBSCRIPTION>
max_messages 1000
return_immediately true
pull_interval 0.5
pull_threads 2
- format json
parse_error_action exception
enable_rpc true
rpc_bind 0.0.0.0
rpc_port 24680
+ <parse>
+ @type json
+ </parse>
</source>
```
- `tag` (required)
- Set tag of messages.
@@ -137,11 +141,9 @@
- If `return_immediately` is `true` or pulling message is stopped by HTTP RPC, this plugin wait `pull_interval` each pull.
- `pull_interval` (optional, default: `5.0`)
- Pulling messages by intervals of specified seconds.
- `pull_threads` (optional, default: `1`)
- Set number of threads to pull messages.
-- `format` (optional, default: `json`)
- - Set input format. See format section in http://docs.fluentd.org/articles/in_tail
- `parse_error_action` (optional, default: `exception`)
- Set error type when parsing messages fails.
- `exception`: Raise exception. Messages are not acknowledged.
- `warning`: Only logging as warning.
- `enable_rpc` (optional, default: `false`)