README.md in remote_syslog-1.4.2 vs README.md in remote_syslog-1.5.0

- old
+ new

@@ -1,46 +1,46 @@ # remote_syslog Ruby daemon & sender -Lightweight Ruby daemon to tail one or more log files and transmit UDP syslog +Lightweight Ruby daemon to tail one or more log files and transmit UDP syslog messages to a remote syslog host (centralized log aggregation). -remote_syslog generates UDP packets itself instead of depending on a system -syslog daemon, so its configuration doesn't affect system-wide +remote_syslog generates UDP packets itself instead of depending on a system +syslog daemon, so its configuration doesn't affect system-wide logging - syslog is just the transport. Uses: * collecting logs from servers & daemons which don't natively support syslog -* when reconfiguring the system logger is less convenient than a +* when reconfiguring the system logger is less convenient than a purpose-built daemon (e.g., automated app deployments) * aggregating files not generated by daemons (e.g., package manager logs) The library can also be used to generate one-off log messages from Ruby code. -Tested with the hosted log management service [Papertrail] and should work for +Tested with the hosted log management service [Papertrail] and should work for transmitting to any syslog server. ## Installation Install the gem, which includes a binary called "remote_syslog": $ [sudo] gem install remote_syslog -Optionally, create a log_files.yml with the log file paths to read and the -host/port to log to (see examples/[log_files.yml.example][sample config]). These can also be +Optionally, create a log_files.yml with the log file paths to read and the +host/port to log to (see examples/[log_files.yml.example][sample config]). These can also be specified as command-line arguments (below). ## Usage $ remote_syslog -h - Usage: remote_syslog [options] <path to add'l log 1> .. <path to add'l log n> + Usage: remote_syslog [options] [<logfile>...] Example: remote_syslog -c configs/logs.yml -p 12345 /var/log/mysqld.log - Options (default): + Options: -c, --configfile PATH Path to config (/etc/log_files.yml) -d, --dest-host HOSTNAME Destination syslog hostname or IP (logs.papertrailapp.com) -p, --dest-port PORT Destination syslog port (514) -D, --no-detach Don't daemonize and detach from the terminal -f, --facility FACILITY Facility (user) @@ -50,30 +50,30 @@ --parse-syslog Parse file as syslog-formatted file -s, --severity SEVERITY Severity (notice) --strip-color Strip color codes --tls Connect via TCP with TLS -h, --help Show this message - + ## Example Typical: $ remote_syslog -Daemonize and collect messages from files listed in `./config/logs.yml` as +Daemonize and collect messages from files listed in `./config/logs.yml` as well as the file `/var/log/mysqld.log`. Send to port `logs.papertrailapp.com:12345`: $ remote_syslog -c configs/logs.yml -p 12345 /var/log/mysqld.log Stay attached to the terminal, look for and use `/etc/log_files.yml` if it exists, write PID to `/tmp/remote_syslog.pid`, and send with facility local0 to `a.server.com:514`: $ remote_syslog -D -d a.server.com -f local0 -P /tmp /var/log/mysqld.log -remote_syslog will daemonize by default. A sample init file is in the gem as +remote_syslog will daemonize by default. A sample init file is in the gem as [remote_syslog.init.d]. You may be able to: $ cp examples/remote_syslog.init.d /etc/init.d/remote_syslog ## Sending messages securely ## @@ -88,25 +88,29 @@ By default, the gem looks for a configuration in /etc/log_files.yml. The gem comes with a [sample config]. Optionally: $ cp examples/log_files.yml.example /etc/log_files.yml - -log_files.yml has filenames to log from (as an array) and hostname and port -to log to (as a hash). Wildcards are supported using * and standard shell -globbing. Filenames given on the command line are additive to those in + +log_files.yml has filenames to log from (as an array) and hostname and port +to log to (as a hash). Wildcards are supported using * and standard shell +globbing. Filenames given on the command line are additive to those in the config file. -Only 1 destination server is supported; the command-line argument wins. +Only 1 destination server is supported; the command-line argument wins. - files: [/var/log/httpd/access_log, /var/log/httpd/error_log, /var/log/mysqld.log, /var/run/mysqld/mysqld-slow.log] + files: + - /var/log/httpd/access_log + - /var/log/httpd/error_log + - /var/log/mysqld.log + - /var/run/mysqld/mysqld-slow.log destination: host: logs.papertrailapp.com port: 12345 -remote_syslog sends the name of the file without a path ("mysqld.log") as -the syslog tag (program name). RFCs 3164 and 5424 limit the tag to 32 +remote_syslog sends the name of the file without a path ("mysqld.log") as +the syslog tag (program name). RFCs 3164 and 5424 limit the tag to 32 characters. Longer filenames are truncated to 32 characters. ## Advanced Configuration (Optional) Here's an [advanced config] which uses all options. @@ -115,62 +119,78 @@ Provide `--hostname somehostname` or use the `hostname` configuration option: hostname: somehostname +### Verify server certificate + +Provide the public key for the remote host when using TLS: + + ssl_server_cert: syslog.crt + + +### Use a client certificate + +Provide a client certificate when connecting via TLS: + + ssl_client_cert_chain: syslog_client.crt + ssl_client_private_key: syslog_client.key + + ### Multiple instances -Run multiple instances to support more than one message-specific file format +Run multiple instances to support more than one message-specific file format or to specify unique syslog hostnames. To do that, provide an alternate PID filename as a command-line option to the additional instance(s). For example: --pid-file remote_syslog_2.pid + ### Parse fields from log messages -Rarely needed. Usually only used when remote_syslog is watching files +Rarely needed. Usually only used when remote_syslog is watching files generated by syslogd (rather than by apps), like ``/var/log/messages``. -remote_syslog can parse the program and hostname from the log line. When one -file contains logs from multiple programs (like with syslog), the log line -may include text that is not part of the log message, like a timestamp, -hostname, or program name. remote_syslog will extract those and use them in +remote_syslog can parse the program and hostname from the log line. When one +file contains logs from multiple programs (like with syslog), the log line +may include text that is not part of the log message, like a timestamp, +hostname, or program name. remote_syslog will extract those and use them in the corresponding syslog packet fields. -To do that, use the config file option `parse_fields` with the name of a +To do that, use the config file option `parse_fields` with the name of a format supported by remote_syslog, or your own regex. Included format names are `syslog` and `rfc3339`. For example: parse_fields: syslog -The included `syslog` format uses the regex `(\w+ \d+ \S+) (\S+) ([^:]+): (.*)` +The included `syslog` format uses the regex `(\w+ \d+ \S+) (\S+) ([^:]+): (.*)` to parse standard syslog lines like this: Jul 18 08:25:08 hostname programname[1234]: The log message -The included `rfc3339` format uses the regex `(\S+) (\S+) ([^: ]+):? (.*)` to +The included `rfc3339` format uses the regex `(\S+) (\S+) ([^: ]+):? (.*)` to parse syslog lines with high-precision RFC 3339 timestamps, like this: 2011-07-16T08:25:08.651413-07:00 hostname programname[1234]: The log message -To parse a format other than those, provide your own regex. It should include -4 backreferences to parse, in order: timestamp, system name, program name, +To parse a format other than those, provide your own regex. It should include +4 backreferences to parse, in order: timestamp, system name, program name, message. -Match and return empty strings for any empty positions where the log line +Match and return empty strings for any empty positions where the log line doesn't provide a value. For example, given the log message: something-meaningless The log message One could use a regex to ignore "something-meaningless" (and not to extract -a program or hostname). To ignore that prefix and return 3 empty values +a program or hostname). To ignore that prefix and return 3 empty values then the log message, use parse_fields with this regex: parse_fields: "something-meaningless ()()()(.*)" -Per-file regexes are not supported. Run multiple instances with different +Per-file regexes are not supported. Run multiple instances with different config files. ## Reporting bugs