docs/index.asciidoc in logstash-input-elasticsearch-4.9.3 vs docs/index.asciidoc in logstash-input-elasticsearch-4.10.0
- old
+ new
@@ -81,12 +81,22 @@
==== Authorization
Authorization to a secure Elasticsearch cluster requires `read` permission at index level and `monitoring` permissions at cluster level.
The `monitoring` permission at cluster level is necessary to perform periodic connectivity checks.
+[id="plugins-{type}s-{plugin}-ecs"]
+==== Compatibility with the Elastic Common Schema (ECS)
+
+When ECS compatibility is disabled, `docinfo_target` uses the `"@metadata"` field as a default, with ECS enabled the plugin
+uses a naming convention `"[@metadata][input][elasticsearch]"` as a default target for placing document information.
+
+The plugin logs a warning when ECS is enabled and `target` isn't set.
+
+TIP: Set the `target` option to avoid potential schema conflicts.
+
[id="plugins-{type}s-{plugin}-options"]
-==== Elasticsearch Input Configuration Options
+==== Elasticsearch Input configuration options
This plugin supports the following configuration options plus the <<plugins-{type}s-{plugin}-common-options>> described later.
[cols="<,<,<",options="header",]
|=======================================================================
@@ -97,10 +107,11 @@
| <<plugins-{type}s-{plugin}-cloud_id>> |<<string,string>>|No
| <<plugins-{type}s-{plugin}-connect_timeout_seconds>> | <<number,number>>|No
| <<plugins-{type}s-{plugin}-docinfo>> |<<boolean,boolean>>|No
| <<plugins-{type}s-{plugin}-docinfo_fields>> |<<array,array>>|No
| <<plugins-{type}s-{plugin}-docinfo_target>> |<<string,string>>|No
+| <<plugins-{type}s-{plugin}-ecs_compatibility>> |<<string,string>>|No
| <<plugins-{type}s-{plugin}-hosts>> |<<array,array>>|No
| <<plugins-{type}s-{plugin}-index>> |<<string,string>>|No
| <<plugins-{type}s-{plugin}-password>> |<<password,password>>|No
| <<plugins-{type}s-{plugin}-proxy>> |<<uri,uri>>|No
| <<plugins-{type}s-{plugin}-query>> |<<string,string>>|No
@@ -195,29 +206,31 @@
index => "mydata-2018.09.*"
query => '{ "query": { "query_string": { "query": "*" } } }'
size => 500
scroll => "5m"
docinfo => true
+ docinfo_target => "[@metadata][doc]"
}
}
output {
elasticsearch {
- index => "copy-of-production.%{[@metadata][_index]}"
- document_type => "%{[@metadata][_type]}"
- document_id => "%{[@metadata][_id]}"
+ index => "copy-of-production.%{[@metadata][doc][_index]}"
+ document_type => "%{[@metadata][doc][_type]}"
+ document_id => "%{[@metadata][doc][_id]}"
}
}
If set, you can use metadata information in the <<plugins-{type}s-{plugin}-add_field>> common option.
Example
[source, ruby]
input {
elasticsearch {
docinfo => true
+ docinfo_target => "[@metadata][doc]"
add_field => {
- identifier => "%{[@metadata][_index]}:%{[@metadata][_type]}:%{[@metadata][_id]}"
+ identifier => "%{[@metadata][doc][_index]}:%{[@metadata][doc][_type]}:%{[@metadata][doc][_id]}"
}
}
}
@@ -234,16 +247,30 @@
[id="plugins-{type}s-{plugin}-docinfo_target"]
===== `docinfo_target`
* Value type is <<string,string>>
- * Default value is `"@metadata"`
+ * Default value depends on whether <<plugins-{type}s-{plugin}-ecs_compatibility>> is enabled:
+ ** ECS Compatibility disabled: `"@metadata"`
+ ** ECS Compatibility enabled: `"[@metadata][input][elasticsearch]"`
-If document metadata storage is requested by enabling the `docinfo`
-option, this option names the field under which to store the metadata
-fields as subfields.
+If document metadata storage is requested by enabling the `docinfo` option,
+this option names the field under which to store the metadata fields as subfields.
+[id="plugins-{type}s-{plugin}-ecs_compatibility"]
+===== `ecs_compatibility`
+
+ * Value type is <<string,string>>
+ * Supported values are:
+ ** `disabled`: CSV data added at root level
+ ** `v1`,`v8`: Elastic Common Schema compliant behavior
+ * Default value depends on which version of Logstash is running:
+ ** When Logstash provides a `pipeline.ecs_compatibility` setting, its value is used as the default
+ ** Otherwise, the default value is `disabled`
+
+Controls this plugin's compatibility with the {ecs-ref}[Elastic Common Schema (ECS)].
+
[id="plugins-{type}s-{plugin}-hosts"]
===== `hosts`
* Value type is <<array,array>>
* There is no default value for this setting.
@@ -400,6 +427,6 @@
[id="plugins-{type}s-{plugin}-common-options"]
include::{include_path}/{type}.asciidoc[]
-:default_codec!:
+:no_codec!: