README.md in embulk-output-bigquery-0.4.3 vs README.md in embulk-output-bigquery-0.4.4
- old
+ new
@@ -59,20 +59,22 @@
| column_options | hash | optional | | See [Column Options](#column-options) |
| default_timezone | string | optional | UTC | |
| default_timestamp_format | string | optional | %Y-%m-%d %H:%M:%S.%6N | |
| payload_column | string | optional | nil | See [Formatter Performance Issue](#formatter-performance-issue) |
| payload_column_index | integer | optional | nil | See [Formatter Performance Issue](#formatter-performance-issue) |
-| gcs_bucket | stringr | optional | nil | See [GCS Bucket](#gcs-bucket) |
+| gcs_bucket | string | optional | nil | See [GCS Bucket](#gcs-bucket) |
| auto_create_gcs_bucket | boolean | optional | false | See [GCS Bucket](#gcs-bucket) |
| progress_log_interval | float | optional | nil (Disabled) | Progress log interval. The progress log is disabled by nil (default). NOTE: This option may be removed in a future because a filter plugin can achieve the same goal |
Client or request options
| name | type | required? | default | description |
|:-------------------------------------|:------------|:-----------|:-------------------------|:-----------------------|
-| timeout_sec | integer | optional | 300 | Seconds to wait for one block to be read |
| open_timeout_sec | integer | optional | 300 | Seconds to wait for the connection to open |
+| timeout_sec | integer | optional | 300 | Seconds to wait for one block to be read (google-api-ruby-client < v0.11.0) |
+| send_timeout_sec | integer | optional | 300 | Seconds to wait to send a request (google-api-ruby-client >= v0.11.0) |
+| read_timeout_sec | integer | optional | 300 | Seconds to wait to read a response (google-api-ruby-client >= v0.11.0) |
| retries | integer | optional | 5 | Number of retries |
| application_name | string | optional | "Embulk BigQuery plugin" | User-Agent |
| sdk_log_level | string | optional | nil (WARN) | Log level of google api client library |
Options for intermediate local files
@@ -353,10 +355,10 @@
prevent_duplicate_insert: true
```
### GCS Bucket
-This is useful to reduce number of consumed jobs, which is limited by [10,000 jobs per project per day](https://cloud.google.com/bigquery/quota-policy#import).
+This is useful to reduce number of consumed jobs, which is limited by [50,000 jobs per project per day](https://cloud.google.com/bigquery/quota-policy#import).
This plugin originally loads local files into BigQuery in parallel, that is, consumes a number of jobs, say 24 jobs on 24 CPU core machine for example (this depends on embulk parameters such as `min_output_tasks` and `max_threads`).
BigQuery supports loading multiple files from GCS with one job (but not from local files, sigh), therefore, uploading local files to GCS and then loading from GCS into BigQuery reduces number of consumed jobs.