README.md in semantic_logger-0.9.0 vs README.md in semantic_logger-0.10.0

- old
+ new

@@ -17,11 +17,14 @@ while the program is running to get more detailed logging in production for just that class Tagged Logging * Supply custom data to be added to every log entry within a block of code, - including libraries and existing Gems + including libraries and existing gems +* Tagged logging is critical for any high traffic site so that one can narrow + down log entries for a single call that is mixed in with log entries + from hundreds of other log entries High Performance * Logging is performed in a separate thread so as not to impact performance of running code @@ -55,14 +58,16 @@ Rails 2 & 3 Support * Just include the semantic_logger gem into Rails and it will immediately replace the existing loggers to improve performance and information in the log files +* The Rails 3 Tagged logging feature is already available for Rails 2 by use Semantic Logger Thread Aware * Includes the process and thread id information in every log entry +* If running JRuby it will also include the name of the thread for every log entry Trace Level * :trace is a new level common in other languages and is commonly used for logging trace level detail. It is intended for logging data at level below @@ -74,12 +79,12 @@ Multiple Destinations * Log to multiple destinations at the same time ( File and MongoDB, etc.. ) * Each destination can also have its own log level. - For example only write :info and above to MongoDB - Or have a second log file for :warn and above log entries + For example, only log :info and above to MongoDB, or :warn and above to a + second log file Benchmarking * The performance of any block of code can be measured and logged at the same time depending on the active log level @@ -89,16 +94,16 @@ * With Semantic Logger it is simple to mix-in additional semantic information with every log entry * The application or class name is automatically included for every log entry under a specific logging instance * Includes the duration of blocks of code -* any Hash containing context specific information such as user_id or location information +* Any hash containing context specific information such as user_id or location information Beyond Tagged Logging * Supply entire hash of custom data to be added to the payload of every log entry - within a block of code, including libraries and existing Gems + within a block of code, including libraries and existing gems NOSQL Destinations * Every log entry is broken down into elements that NOSQL data stores can understand: @@ -128,24 +133,24 @@ * Tagged logging keeps any tagging data on a per-thread basis to ensure that tags from different threads are not inter-mingled ### Introduction -SemanticLogger is a Logger that supports logging of meta-data, along with text messages +Semantic Logger is a Logger that supports logging of meta-data, along with text messages to multiple appenders An appender is a Logging destination such as a File, MongoDB collection, etc.. Multiple Appenders can be active at the same time. All log entries are written to each appender. Machines can understand the logged data without having to use complex Regular Expressions or other text parsing techniques -SemanticLogger, sits on top of existing logger implementations and can also +Semantic Logger, sits on top of existing logger implementations and can also be used as a drop in replacement for existing Ruby loggers. This allows the existing logging to be replaced immediately with the -SemanticLogger Appenders, and over time the calls can be replaced with ones +Semantic Logger Appenders, and over time the calls can be replaced with ones that contain the necessary meta-data. Example of current calls: ```ruby @@ -173,11 +178,11 @@ ```javascript db.logs.find({"payload.table":"users", "payload.action":"query", "payload.duration":{$gt:100} }) ``` -Since SemanticLogger can call existing Loggers, it does not force end-users +Since Semantic Logger can call existing Loggers, it does not force end-users to have to adopt a Semantic aware adapter. Although, such adapters create tremendous value in the problem monitoring and determination processes. ### Logging API @@ -290,11 +295,11 @@ - message: The mandatory text message to log. - params: ``` :log_exception Control whether or how an exception thrown in the block is - reported by SemanticLogger. Values: + reported by Semantic Logger. Values: :full Log the exception class, message, and backtrace :partial Log the exception class and messag The backtrace will not be logged @@ -328,11 +333,11 @@ If only the rails logger is being used, then :trace level calls will be logged as debug calls only if the log level is set to trace #### Changing the Class name for Log Entries -When Semantic Logger is included on a Rails project it automatically replaces the +When Semantic Logger is included in a Rails project it automatically replaces the loggers for Rails, ActiveRecord::Base, ActionController::Base, and ActiveResource::Base with wrappers that set their Class name. For example in semantic_logger/railtie.rb: ```ruby ActiveRecord::Base.logger = SemanticLogger::Logger.new(ActiveRecord) @@ -341,28 +346,25 @@ By replacing their loggers we now get the class name in the text logging output: 2012-08-30 15:24:13.439 D [47900:main] ActiveRecord -- SQL (12.0ms) SELECT `schema_migrations`.`version` FROM `schema_migrations` It is recommended to include a class specific logger for all major classes that will -be logging. For Example: +be logging using the SemanticLogger::Attribute mix-in. For Example: ```ruby -require 'sync_attr' require 'semantic_logger' class ExternalSupplier - # Gem sync_attr is a dependency of semantic_logger so is already installed - include SyncAttr + # Lazy load logger class variable on first use + include SemanticLogger::Attribute - # Lazy initializes the class logger on it's first call in a thread-safe way - sync_cattr_reader :logger do - SemanticLogger::Logger.new(self) - end + def call_supplier(amount, name) + logger.debug "Calculating with amount", { :amount => amount, :name => name } - def call(params) - self.class.logger.benchmark_info "Calling external interface" do - # Code to call external service ... + # Measure and log on completion how long the call took to the external supplier + logger.benchmark_info "Calling external interface" do + # Code to call the external supplier ... end end end ``` @@ -425,10 +427,30 @@ logger.debug("Hello World") # ... end ``` +### Using SemanticLogger outside of Rails + +Example: + +```ruby +require 'semantic_logger' + +# Set the log level to log everything +SemanticLogger::Logger.default_level = :trace + +# Add a file appender to log everything to a file +SemanticLogger::Logger.appenders << SemanticLogger::Appender::File.new('dev.log') + +# Add an appender to only log :info and above to standard out +SemanticLogger::Logger.appenders << SemanticLogger::Appender::File.new(STDOUT, :info) + +logger = SemanticLogger::Logger.new('Example') +logger.info "Hello World" +``` + ### Configuration The Semantic Logger follows the principle where multiple appenders can be active at the same time. This allows one to log to MongoDB and the Rails ActiveResource::BufferedLogger at the same time. @@ -475,50 +497,89 @@ :db => db, :collection_size => 25.gigabytes ) end ``` +### Log Struct +Internally all log messages are passed around in a Log Struct. In order +to write your own custom formatter or log appender it is necessary to understand +the fields: + +```ruby +Log = Struct.new(:level, :thread_name, :name, :message, :payload, :time, :duration, :tags, :level_index) +``` +level [Symbol] + +* Log level of the supplied log call +* :trace, :debug, :info, :warn, :error, :fatal + +thread_name [String] + +* Name or id of the thread in which the logging call was called + +name [String] + +* Class name supplied to the logging instance + +message [String] + +* Text message to be logged + +payload [Hash|Exception] + +* Optional Hash or Ruby Exception object to be logged + +time [Time] + +* The time at which the log entry was created + +duration [Float] + +* The time taken in milli-seconds to complete a benchmark call + +tags [Array<String>] + +* Any tags active on the thread when the log call was made + +level_index [Integer] + +* Internal use only. Index of the log level + #### Custom Formatters The formatting for each appender can be replaced with custom code. To replace the existing formatter supply a block of code when creating the appender. -For example to replace the Rails or Ruby text log formatter, in the environment configuration file: +Example: Formatter that just returns the Log Struct ```ruby +require 'semantic_logger' + +SemanticLogger::Logger.default_level = :trace + +appender = SemanticLogger::Appender::File.new(STDOUT) do |log| + # This formatter just returns the log struct as a string + log.inspect +end + +SemanticLogger::Logger.appenders << appender + +logger = SemanticLogger::Logger.new('Hello') +logger.info "Hello World" +``` +Output: + + #<struct SemanticLogger::Base::Log level=:info, thread_name=70167090649820, name="Hello", message="Hello World", payload=nil, time=2012-10-24 10:09:33 -0400, duration=nil, tags=nil, level_index=2> + + +Example: Replace the Rails log formatter, in the environment configuration file: + +```ruby config.after_initialize do # Since the Rails logger is already initialized, replace its default formatter config.semantic_logger.appenders.first.formatter = Proc.new do |log| - # log is a struct with the following fields: - # - # level - # Log level of the supplied log call - # :trace, :debug, :info, :warn, :error, :fatal - # - # thread_name - # Name of the thread in which the logging call was called - # - # name - # Class name supplied to the logging instance - # - # message - # Text message to be logged - # - # payload - # Optional Hash or Ruby Exception object to be logged - # - # time - # The time at which the log entry was created - # - # duration - # The time taken to complete a benchmark call - # - # tags - # Any tags active on the thread when the log call was made - # - message = log.message.to_s tags = log.tags.collect { |tag| "[#{tag}]" }.join(" ") + " " if log.tags && (log.tags.size > 0) if log.payload if log.payload.is_a?(Exception) @@ -534,46 +595,19 @@ str end end ``` -For example to replace the MongoDB formatter, in the environment configuration file: +Example: Replace the MongoDB formatter, in the environment configuration file: ```ruby config.after_initialize do # Log to MongoDB and supply a custom document formatter mongodb_appender = SemanticLogger::Appender::MongoDB.new( :db => Cache::Work.db, :collection_size => 25.gigabytes ) do |log| - # log is a struct with the following fields: - # level - # Log level of the supplied log call - # :trace, :debug, :info, :warn, :error, :fatal - # - # thread_name - # Name of the thread in which the logging call was called - # - # name - # Class name supplied to the logging instance - # - # message - # Text message to be logged - # - # payload - # Optional Hash or Ruby Exception object to be logged - # - # time - # The time at which the log entry was created - # - # duration - # The time taken to complete a benchmark call - # - # tags - # Any tags active on the thread when the log call was made - # - # Return a document (Hash) of the data to be saved to MongoDB document = { :time => log.time, :host_name => SemanticLogger::Appender::MongoDB.host_name, :pid => $PID, @@ -677,51 +711,12 @@ * In the initializer connect to the resource being logged to * Implement #log(log) which needs to write to the relevant resource * Implement #flush if the resource can be flushed * Write a test for the new appender -The #log method takes the log struct as a parameter which is defined as follows: -```ruby -Log = Struct.new(:level, :thread_name, :name, :message, :payload, :time, :duration, :tags, :level_index) -``` -level +The #log method takes the log struct as a parameter which is described above. -* Log level of the supplied log call -* :trace, :debug, :info, :warn, :error, :fatal - -thread_name - -* Name or id of the thread in which the logging call was called - -name - -* Class name supplied to the logging instance - -message - -* Text message to be logged - -payload [Hash|Exception] - -* Optional Hash or Ruby Exception object to be logged - -time [Time] - -* The time at which the log entry was created - -duration [Float] - -* The time taken in milli-seconds to complete a benchmark call - -tags [Array<String>] - -* Any tags active on the thread when the log call was made - -level_index - -* Internal use only. Index of the log level - Basic outline for an Appender: ```ruby require 'semantic_logger' @@ -732,11 +727,11 @@ end # Display the log struct and the text formatted output def log(log) p log - puts @formatter.call(log) + puts formatter.call(log) end # Optional def flush puts "Flush :)" @@ -771,18 +766,21 @@ ### Install gem install semantic_logger -To log to MongoDB +To log to MongoDB, it also needs the Ruby Mongo Driver gem install mongo ### Future +- In V1: Move Railtie to it's own gem so that the Rails logger is not replaced + automatically +- In V1: Add support for a configuration file that can set log level by class name - Configuration file to support setting the log level for a specific class - Configuration file to support adding appenders -- Based on demand add direct appenders for: Syslog, hadoop, redis +- Based on demand add appenders for: Syslog, hadoop, redis Development ----------- Want to contribute to Semantic Logger?