= ntail

A tail(1)-like utility for nginx log files that supports parsing, filtering and formatting individual log lines.

  > gem install ntail

== Examples

* read from STDIN and print each line to STDOUT <em>(stop with ^D)</em>

    > ntail

* process an nginx log file and print each line to STDOUT

    > ntail /var/log/nginx/access.log

* tail an "active" nginx log file and print each new line to STDOUT <em>(stop with ^C)</em>

    > tail -f /var/log/nginx/access.log | ntail

* tail STDIN and print out the length of each line <em>(to illustrate -e option)</em>

    > ntail -e '{ |line| puts line.size }'

* tail STDIN but only print out non-empty lines <em>(to illustrate -f option)</em>

    > ntail -f '{ |line| line.size > 0 }'

* the following invocations behave exactly the same <em>(to illustrate -e and -f options)</em>

    > ntail
    > ntail -f '{ |line| true }' -e '{ |line| puts line }'

* print out all HTTP requests that are coming from a given IP address

    > ntail -f '{ |line| line.remote_address == "208.67.222.222" }' /var/log/nginx/access.log

* find all HTTP requests that resulted in a '5xx' HTTP error/status code <em>(e.g. Rails 500 errors)</em>

    > gunzip -S .gz -c access.log-20101216.gz | ntail -f '{ |line| line.server_error_status? }'

* generate a summary report of HTTP status codes, for all non-200 HTTP requests

    > ntail -f '{ |line| line.status != "200" }' -e '{ |line| puts line.status }' access.log | sort | uniq -c
    76 301
    16 302
     2 304
     1 406

* print out GeoIP country and city information for each HTTP request <em>(depends on the optional <tt>geoip</tt> gem)</em>

    > ntail -e '{ |line| puts [line.to_country_s, line.to_city_s].join("\t") }' /var/log/nginx/access.log
    United States   Los Angeles
    United States   Houston
    Germany         Berlin
    United Kingdom  London

* print out the IP address and the corresponding host name for each HTTP request <em>(slows things down considerably, due to <tt>nslookup</tt> call)</em>

    > ntail -e '{ |line| puts [line.remote_address, line.to_host_name].join("\t") }' /var/log/nginx/access.log
    66.249.72.196   crawl-66-249-72-196.googlebot.com
    67.192.120.134  s402.pingdom.com
    75.31.109.144   adsl-75-31-109-144.dsl.irvnca.sbcglobal.net

== TODO

* implement a native <tt>"-f"</tt> option for ntail, similar to that of <tt>tail(1)</tt>
* implement a <tt>"-i"</tt> option ("ignore exceptions"/"continue processing"), if handling a single line raised an exception

* make <tt>PROXY_IP_ADDRESS</tt> configurable (from command line and/or rc file)
* make <tt>OFFICE_IP_ADDRESS</tt> configurable (from command line and/or rc file)
* make <tt>KNOWN_SEARCH_BOTS</tt> configurable (from command line and/or rc file)
* make <tt>INTERNAL_REFERERS</tt> configurable (from command line and/or rc file)
* make <tt>AUTOMATED_REQUESTS</tt> configurable (from command line and/or rc file)
* make <tt>STATIC_REPOS</tt> configurable (from command line and/or rc file)

== Acknowledgements

ntail's parsing feature is inspired by an nginx log parser written by {Richard Taylor (moomerman)}[https://github.com/moomerman]

== Contributing to ntail
 
* Check out the latest master to make sure the feature hasn't been implemented or the bug hasn't been fixed yet
* Check out the issue tracker to make sure someone already hasn't requested it and/or contributed it
* Fork the project
* Start a feature/bugfix branch
* Commit and push until you are happy with your contribution
* Make sure to add tests for it. This is important so I don't break it in a future version unintentionally.
* Please try not to mess with the Rakefile, version, or history. If you want to have your own version, or is otherwise necessary, that is fine, but please isolate to its own commit so I can cherry-pick around it.

== Copyright

Copyright (c) 2010 Peter Vandenberk. See LICENSE.txt for further details.