README.md in embulk-input-splunk-0.1.2 vs README.md in embulk-input-splunk-0.1.3

- old
+ new

@@ -22,19 +22,27 @@ - **password**: splunk password (string, required) - **port**: splunk API port (integer, default: 8089) - **query**: the query you wish to run. It should be prefixed with "search" (string required) - **earliest_time**: the earliest time for the splunk search. (string, default: nil, which is unbounded) - **latest_time**: the latest time for the splunk search. (string, default: nil, which is unbounded) +- **incremental**: whether to resume next search from last result time (boolean, default: false) ### Earliest and latest times Splunk's required data format is `%Y-%m-%dT%H:%M:%S.%L%:z` which is the required format for `earliest_time` and `latest_time`. In addition, Splunk relative time operations are also accepted, such as -1d@d. For more information, see the [Splunk documentation](https://docs.splunk.com/Documentation/Splunk/7.0.2/SearchReference/SearchTimeModifiers) +### Incremental loads + +Incremental support is basic. The logic is: + +- always rely on `_time` field in Splunk +- determine latest `_time` in search +- use latest `_time` as `earliest_time` in next run + ### Number of returned results The default Splunk API limits resuts to 100. In this plugin, the limit is not set, so it is possible to generate very large result sets. To limit the number of results, use the `head` or `tail` command in your query. - ## Examples Remember the queries much be prefixed with the search command or they are unlikely not to work.