docs/index.asciidoc in logstash-input-elasticsearch-4.9.3 vs docs/index.asciidoc in logstash-input-elasticsearch-4.10.0

- old
+ new

@@ -81,12 +81,22 @@ ==== Authorization Authorization to a secure Elasticsearch cluster requires `read` permission at index level and `monitoring` permissions at cluster level. The `monitoring` permission at cluster level is necessary to perform periodic connectivity checks. +[id="plugins-{type}s-{plugin}-ecs"] +==== Compatibility with the Elastic Common Schema (ECS) + +When ECS compatibility is disabled, `docinfo_target` uses the `"@metadata"` field as a default, with ECS enabled the plugin +uses a naming convention `"[@metadata][input][elasticsearch]"` as a default target for placing document information. + +The plugin logs a warning when ECS is enabled and `target` isn't set. + +TIP: Set the `target` option to avoid potential schema conflicts. + [id="plugins-{type}s-{plugin}-options"] -==== Elasticsearch Input Configuration Options +==== Elasticsearch Input configuration options This plugin supports the following configuration options plus the <<plugins-{type}s-{plugin}-common-options>> described later. [cols="<,<,<",options="header",] |======================================================================= @@ -97,10 +107,11 @@ | <<plugins-{type}s-{plugin}-cloud_id>> |<<string,string>>|No | <<plugins-{type}s-{plugin}-connect_timeout_seconds>> | <<number,number>>|No | <<plugins-{type}s-{plugin}-docinfo>> |<<boolean,boolean>>|No | <<plugins-{type}s-{plugin}-docinfo_fields>> |<<array,array>>|No | <<plugins-{type}s-{plugin}-docinfo_target>> |<<string,string>>|No +| <<plugins-{type}s-{plugin}-ecs_compatibility>> |<<string,string>>|No | <<plugins-{type}s-{plugin}-hosts>> |<<array,array>>|No | <<plugins-{type}s-{plugin}-index>> |<<string,string>>|No | <<plugins-{type}s-{plugin}-password>> |<<password,password>>|No | <<plugins-{type}s-{plugin}-proxy>> |<<uri,uri>>|No | <<plugins-{type}s-{plugin}-query>> |<<string,string>>|No @@ -195,29 +206,31 @@ index => "mydata-2018.09.*" query => '{ "query": { "query_string": { "query": "*" } } }' size => 500 scroll => "5m" docinfo => true + docinfo_target => "[@metadata][doc]" } } output { elasticsearch { - index => "copy-of-production.%{[@metadata][_index]}" - document_type => "%{[@metadata][_type]}" - document_id => "%{[@metadata][_id]}" + index => "copy-of-production.%{[@metadata][doc][_index]}" + document_type => "%{[@metadata][doc][_type]}" + document_id => "%{[@metadata][doc][_id]}" } } If set, you can use metadata information in the <<plugins-{type}s-{plugin}-add_field>> common option. Example [source, ruby] input { elasticsearch { docinfo => true + docinfo_target => "[@metadata][doc]" add_field => { - identifier => "%{[@metadata][_index]}:%{[@metadata][_type]}:%{[@metadata][_id]}" + identifier => "%{[@metadata][doc][_index]}:%{[@metadata][doc][_type]}:%{[@metadata][doc][_id]}" } } } @@ -234,16 +247,30 @@ [id="plugins-{type}s-{plugin}-docinfo_target"] ===== `docinfo_target` * Value type is <<string,string>> - * Default value is `"@metadata"` + * Default value depends on whether <<plugins-{type}s-{plugin}-ecs_compatibility>> is enabled: + ** ECS Compatibility disabled: `"@metadata"` + ** ECS Compatibility enabled: `"[@metadata][input][elasticsearch]"` -If document metadata storage is requested by enabling the `docinfo` -option, this option names the field under which to store the metadata -fields as subfields. +If document metadata storage is requested by enabling the `docinfo` option, +this option names the field under which to store the metadata fields as subfields. +[id="plugins-{type}s-{plugin}-ecs_compatibility"] +===== `ecs_compatibility` + + * Value type is <<string,string>> + * Supported values are: + ** `disabled`: CSV data added at root level + ** `v1`,`v8`: Elastic Common Schema compliant behavior + * Default value depends on which version of Logstash is running: + ** When Logstash provides a `pipeline.ecs_compatibility` setting, its value is used as the default + ** Otherwise, the default value is `disabled` + +Controls this plugin's compatibility with the {ecs-ref}[Elastic Common Schema (ECS)]. + [id="plugins-{type}s-{plugin}-hosts"] ===== `hosts` * Value type is <<array,array>> * There is no default value for this setting. @@ -400,6 +427,6 @@ [id="plugins-{type}s-{plugin}-common-options"] include::{include_path}/{type}.asciidoc[] -:default_codec!: +:no_codec!: