index.asciidoc in logstash-input-file-4.1.4

- old
+ new

@@ -76,16 +76,10 @@
 Read mode also allows for an action to take place after processing the file completely.
 
 In the past attempts to simulate a Read mode while still assuming infinite streams
 was not ideal and a dedicated Read mode is an improvement.
 
-==== Reading from remote network volumes
-
-The file input is not tested on remote filesystems such as NFS, Samba, s3fs-fuse, etc. These
-remote filesystems typically have behaviors that are very different from local filesystems and
-are therefore unlikely to work correctly when used with the file input.
-
 ==== Tracking of current position in watched files
 
 The plugin keeps track of the current position in each file by
 recording it in a separate file named sincedb. This makes it
 possible to stop and restart Logstash and have it pick up where it
@@ -101,10 +95,14 @@
 
 A different `sincedb_path` must be used for each input. Using the same
 path will cause issues. The read checkpoints for each input must be
 stored in a different path so the information does not override.
 
+Files are tracked via an identifier. This identifier is made up of the
+inode, major device number and minor device number. In windows, a different
+identifier is taken from a `kernel32` API call.
+
 Sincedb records can now be expired meaning that read positions of older files
 will not be remembered after a certain time period. File systems may need to reuse
 inodes for new content. Ideally, we would not use the read position of old content,
 but we have no reliable way to detect that inode reuse has occurred. This is more
 relevant to Read mode where a great many files are tracked in the sincedb.
@@ -121,26 +119,38 @@
 old sincedb records converted to the new format, this is blank.
 
 On non-Windows systems you can obtain the inode number of a file
 with e.g. `ls -li`.
 
+==== Reading from remote network volumes
+
+The file input is not thoroughly tested on remote filesystems such as NFS,
+Samba, s3fs-fuse, etc, however NFS is occasionally tested. The file size as given by
+the remote FS client is used to govern how much data to read at any given time to
+prevent reading into allocated but yet unfilled memory.
+As we use the device major and minor in the identifier to track "last read"
+positions of files and on remount the device major and minor can change, the
+sincedb records may not match across remounts.
+Read mode might not be suitable for remote filesystems as the file size at
+discovery on the client side may not be the same as the file size on the remote side
+due to latency in the remote to client copy process.
+
 ==== File rotation in Tail mode
 
 File rotation is detected and handled by this input, regardless of
 whether the file is rotated via a rename or a copy operation. To
 support programs that write to the rotated file for some time after
 the rotation has taken place, include both the original filename and
 the rotated filename (e.g. /var/log/syslog and /var/log/syslog.1) in
-the filename patterns to watch (the `path` option). Note that the
-rotated filename will be treated as a new file so if
-`start_position` is set to 'beginning' the rotated file will be
-reprocessed.
-
-With the default value of `start_position` ('end') any messages
-written to the end of the file between the last read operation prior
-to the rotation and its reopening under the new name (an interval
-determined by the `stat_interval` and `discover_interval` options)
-will not get picked up.
+the filename patterns to watch (the `path` option).
+For a rename, the inode will be detected as having moved from
+`/var/log/syslog` to `/var/log/syslog.1` and so the "state" is moved
+internally too, the old content will not be reread but any new content
+on the renamed file will be read.
+For copy/truncate the copied content into a new file path, if discovered, will
+be treated as a new discovery and be read from the beginning. The copied file
+paths should therefore not be in the filename patterns to watch (the `path` option).
+The truncation will be detected and the "last read" position updated to zero.
 
 [id="plugins-{type}s-{plugin}-options"]
 ==== File Input Configuration Options
 
 This plugin supports the following configuration options plus the <<plugins-{type}s-{plugin}-common-options>> described later.