= fluent-plugin-rewrite-tag-filter

== Overview

=== RewriteTagFilterOutput

It has designed to rewrite tags like mod_rewrite.
Also you can change a tag from apache log by domain, status-code(ex. 500 error), 
user-agent, and request-uri and so on with regular expression. 

== Installation

    # native fluentd user
    gem install fluent-plugin-rewrite-tag-filter
    
    # td-agent user
    /usr/lib64/fluent/ruby/bin/gem install fluent-plugin-rewrite-tag-filter

== Configuration

Format

    rewruterule<num:1-200> <attribute> <regex_pattern> <new_tag>

Usage

    # It's a sample to exclude some static file log before split tag by domain.
    <source>
      type tail
      path /var/log/httpd/access_log
      format /^(?<domain>[^ ]*) (?<host>[^ ]*) [^ ]* (?<user>[^ ]*) \[(?<time>[^\]]*)\] "(?<method>\S+)(?: +(?<path>[^ ]*) +\S*)?" (?<status>[^ ]*) (?<size>[^ ]*)(?: "(?<referer>[^\"]*)" "(?<agent>[^\"]*)" (?<response_time>[^ ]*))?$/
      time_format %d/%b/%Y:%H:%M:%S %z
      tag td.apache.access
      pos_file /var/log/td-agent/apache_access.pos
    </source>

    <match td.apache.access>
      type rewrite_tag_filter
      rewriterule1 path \.(gif|jpe?g|png|pdf|zip)$ clear
      rewriterule2 domain ^maps\.google\.com$ site.GoogleMap
      rewriterule3 domain ^news\.google\.com$ site.GoogleNews
      rewriterule4 domain ^mail\.google\.com$ site.GoogleMail
      rewriterule5 domain .+ site.unmatched
    </match>

    <match site.*>
      type mongo
      host localhost
      database apache_access
      remove_tag_prefix site
      tag_mapped
      capped
      capped_size 100m
    </match>

    <match clear>
      type null
    </match>

Result

    $ mongo
    MongoDB shell version: 2.0.6
    > use apache_access
    switched to db apache_access
    > show collections
    GoogleMap
    GoogleNews
    GoogleMail
    unmatched

== Example

- Example1: how to analyze response_time, response_code and user_agent for each virtual domain websites.
  - https://github.com/y-ken/fluent-plugin-rewrite-tag-filter/blob/master/example.conf

- Example2: how to exclude specified patterns before analyze response_time for each virtual domain websites.
  - https://github.com/y-ken/fluent-plugin-rewrite-tag-filter/blob/master/example2.conf

== Review

- 自在にタグを書き換える fluent-plugin-rewrite-tag-filter でログ解析が捗るお話 #fluentd
  - http://d.hatena.ne.jp/yoshi-ken/20120701/1341137269

== TODO

- patches welcome!

== Copyright

Copyright:: Copyright (c) 2012- Kentaro YOSHIDA (@yoshi_ken)
License::   Apache License, Version 2.0