[English](https://github.com/fetaro/rbatch/blob/master/README.md "english") | [Japanese ](https://github.com/fetaro/rbatch/blob/master/README.ja.md "japanese") |  [Document (YardDoc)](http://fetaro.github.io/rbatch/index.html)

RBatch: Batch Script Framework
=============

About RBatch
--------------

RBatch is a ruby-based framework for batch scripts. RBatch help to make a batch script such as "data backup" or "proccess start ".

There are following functions.

* Auto Logging
* Auto Library Loading
* Auto Mail Sending
* Auto Config Reading
* External Command Wrapper
* Double Run Check

Note: RBatch works on Ruby 1.9.

Note: This software is released under the MIT License, see LICENSE.txt.

Quick Start
--------------

    $ sudo gem install rbatch
    $ rbatch-init    # => make directories and sample scripts
    $ ruby bin/hello_world.rb
    $ cat log/YYYYMMDD_HHMMSS_hello_world.log

Overview
--------------

### RBatch home directory

When you set `${RB_HOME}` environment variable, RBatch home directory is fixed at `${RB_HOME}`.

When you do NOT set `${RB_HOME}`, `${RB_HOME}` is the parent directory of the directory where the script is located. In other words, default of `${RB_HOME}` is `(script path)/../` .

### Directory Structure and File Naming Convention

Follow the rule of "convention over configuration",
RBach has convention of file naming and directory structure in advance.
If you follow these rules, you will gain plenty development environment without any configurations.

The rules are

* If you make `${RB_HOME}/bin/hoge.rb` as the script...
* individual config file should be `${RB_HOME}/conf/hoge.yaml`
* common(shared) config file should be `${RB_HOME}/conf/common.yaml`
* libraries should be `${RB_HOME}/lib/*.rb`

then

* log files are output at `${RB_HOME}/log/YYYYMMDD_HHMMSS_hoge.log`

For example

    ${RB_HOME}         <--- RBatch home
     |
     |- .rbatchrc      <--- RBatch Run-Conf
     |
     |- bin            <--- Scripts
     |   |-  A.rb
     |   |-  B.rb
     |
     |- conf           <--- Config files
     |   |-  A.yaml    <--- Individual config file
     |   |-  B.yaml
     |   |-  common.yaml <--- Common config file
     |
     |- log            <--- Log files
     |   |-  YYYYMMDD_HHMMSS_A.log
     |   |-  YYYYMMDD_HHMMSS_B.log
     |
     |- lib            <--- Libraries
         |-  lib_X.rb
         |-  lib_Y.rb

### Auto Logging

Use the auto logging block ,`RBatch::Log`, RBatch automatically output logfiles.
The default location of log files are `${RB_HOME}/log/YYYYMMDD_HHMMSS_(script base).log`.
If an exception is raised in auto logging block, RBatch rescue and write the stack trace to the logfile.

sample

make script `${RB_HOME}/bin/sample1.rb`

```ruby
require 'rbatch'
RBatch::Log.new(){ |log|  # Logging block
  log.info "info string"
  log.error "error string"
  raise "exception"
}
```

Run script. Log file is `${RB_HOME}/log/20121020_005953_sample1.log`

    # Logfile created on 2012-10-20 00:59:53 +0900 by logger.rb/25413
    [2012-10-20 00:59:53 +900] [INFO ] info string
    [2012-10-20 00:59:53 +900] [ERROR] error string
    [2012-10-20 00:59:53 +900] [FATAL] Caught exception; existing 1
    [2012-10-20 00:59:53 +900] [FATAL] exception (RuntimeError)
        [backtrace] test.rb:6:in `block in <main>'
        [backtrace] /usr/local/lib/ruby192/lib/ruby/gems/1.9.1/gems/rbatch-1.0.0/lib/rbatch/auto_logger.rb:37:in `initialize'
        [backtrace] test.rb:3:in `new'
        [backtrace] test.rb:3:in `<main>'

### Auto Library Loading

If you make the libraries at `${RB_HOME}/lib/*.rb`, these are loaded(required) before script run.

### Auto Mail Sending

By using `log_send_mail` option, when the logs with ERROR or FATAL level are output, RBatch sends the error messages by e-mail. 

### Auto Config Reading

If you make the configuration file ,`${RB_HOME}/conf/"(script base).yaml"`, this file is read and parsed automatically.

sample

make the config file `${RB_HOME}/conf/sample2.yaml`.

    key: value
    array:
     - item1
     - item2
     - item3

In script named `${RB_HOME}/bin/sample2.rb`, this config values can be used.

```ruby
require 'rbatch'

# You can read config value without loading file.
p RBatch.config["key"]   # => "value" 
p RBatch.config["array"] # => ["item1", "item2", "item3"]

# If key does not exist, raise exception
p RBatch.config["not_exist"] # => Raise RBatch::ConfigException
```

#### Common Config

If you can use the common config file , `${RB_HOME}/conf/common.yaml`, this config values can be used by all scripts.
The name of common config file is changed by option `common_conf_name`.

### External Command Wrapper

`RBatch.cmd` is external command wapper.

This function return a result object which contains command's "STDOUT", "STDERR" and "exit status".

sample

```ruby
require 'rbatch'

r = RBatch.cmd("ls")
p r.stdout # => "fileA\nfileB\n"
p r.stderr # => ""
p r.status # => 0
```

If you want to set a timeout of external command, you can use `cmd_timeout` option.

By using `cmd_raise` option, if exit status of command is not 0, RBatch raise an Exception.
This function help to handle the errors of external command.

### Double Run Check

By using `forbid_double_run` option, two scripts both has same name cannot run at the same time.

Manual
--------------

Manual -> [Document (YardDoc)](http://fetaro.github.io/rbatch/index.html)

Sample
--------------

### AWS EC2 Volume Backup Script

First you make the configuration file.

Config File : `${RB_HOME}/conf/ec2_create_snapshot.yaml`

```
access_key : AKIAITHEXXXXXXXXX
secret_key : JoqJSdP8+tpdFYWljVbG0+XXXXXXXXXXXXXXX

```

Next, you write the script.

Script : `${RB_HOME}/bin/ec2_create_snapshot.rb`

```ruby
require 'rbatch'  # <= require rbatch
require 'aws-sdk'
require 'net/http'

RBatch::Log.new do |log| # <= Start Log block. And write main logic in this block.
  # get ec2 region
  @ec2_region = "ec2." +
    Net::HTTP.get("169.254.169.254", "/latest/meta-data/placement/availability-zone").chop +
    ".amazonaws.com"
  log.info("ec2 region : #{@ec2_region}")  # <= Output Log

  #create ec2 instance
  @ec2 = AWS::EC2.new(:access_key_id     => RBatch.config["access_key"],  # <= Read Config
                      :secret_access_key => RBatch.config["secret_key"],
                      :ec2_endpoint      => @ec2_region)


  # create instance
  @instance_id = Net::HTTP.get("169.254.169.254", "/latest/meta-data/instance-id")
  @instance = @ec2.instances[@instance_id]
  log.info("instance_id : #{@instance_id}")

  # create snapshots
  @instance.block_devices.each do | dev |
    desc = @instance_id + " " + dev[:device_name] + " " +
      dev[:ebs][:volume_id] + " " +Time.now.strftime("%Y/%m/%d %H:%M").to_s
    log.info("create snapshot : #{desc}")
    @ec2.volumes[dev[:ebs][:volume_id]].create_snapshot(desc)
    log.info("sucess")
  end
end

```

Finally you run the script, then following logfile is output.

Log file : `${RB_HOME}/log/20140121_123124_ec2_create_snapshot.log`

```
[2014-01-21 12:31:24 -0500] [INFO ] === START RBatch === (PID=10095)
[2014-01-21 12:31:24 -0500] [INFO ] RB_HOME : "/opt/MyProject"
[2014-01-21 12:31:24 -0500] [INFO ] Load Run-Conf: "/opt/MyProject/.rbatchrc"
[2014-01-21 12:31:24 -0500] [INFO ] Load Config  : "/opt/MyProject/conf/ec2_create_snapshot.yaml"
[2014-01-21 12:31:24 -0500] [INFO ] Start Script : "/opt/MyProject/bin/ec2_create_snapshot.rb"
[2014-01-21 12:31:24 -0500] [INFO ] Logging Start: "/opt/MyProject/log/20140121_123124_ec2_create_snapshot.log"
[2014-01-21 12:31:24 -0500] [INFO ] ec2 region : ec2.ap-northeast-1.amazonaws.com
[2014-01-21 12:31:25 -0500] [INFO ] instance_id : i-cc25f1c9
[2014-01-21 12:31:25 -0500] [INFO ] create snapshot : i-cc25f1c9 /dev/sda1 vol-82483ea7 2014/01/21 12:31
[2014-01-21 12:31:25 -0500] [INFO ] sucess
```

Only you write a short code, this batch script has logging and read-config function.

Customize
--------------
If you want to customize RBatch, you have two methods.

* (1) Write Run-Conf(`${RB_HOME}/.rbatchrc`).
* (2) Pass an option object to constructor of RBatch classes in a script.

When an option is set in both (1) and (2), (2) is prior to (1).

#### Customize by writing Run-Conf (.rbatchrc)

Sample of RBatch Run-Conf `${RB_HOME}/.rbatchrc`.

```
# RBatch Run-Conf (.rbatchrc)
#
#   This format is YAML.
#

# -------------------
# Global setting
# -------------------

# Conf Directory
#
#   Default is "<home>/conf"
#   <home> is replaced to ${RB_HOME}
#
#conf_dir : <home>/config/
#conf_dir : /etc/rbatch/

# Common Config file name
#
#   Default is "common.yaml"
#
#common_conf_name : share.yaml

# Library Directory
#
#   Default is "<home>/lib"
#   <home> is replaced to ${RB_HOME}
#
#lib_dir: /usr/local/lib/rbatch/

# Auto Library Load
#
#   Default is true
#   If true, require "(library directory)/*.rb" before script run.
#
#auto_lib_load : true
#auto_lib_load : false

# Forbit Script Running Doubly
#
#   Default is false.
#   If true, two same name scripts cannot run at the same time.
#
#forbid_double_run: true
#forbid_double_run: false

# RBatch Journal Level
#
#   Default is 1
#   RBatch Journal is message of RBatch and is output to STDOUT. 
#   If 2, put much more information.
#   If 0, put nothing.
#
#   Example of RBatch Journal are follows.
#     [RBatch] === START RBatch === (PID=5795)
#     [RBatch] RB_HOME : "/path/to/"
#     [RBatch] Load Run-Conf: "/path/to/.rbatchrc"
#     [RBatch] Start Script : "/path/to/bin/hello.rb"
#     ....
#
#rbatch_journal_level : 2
#rbatch_journal_level : 0

# Mix RBatch Journal to Logs
#
#   Default is true.
#   If true, RBatch Journal is output not only STDOUT
#   but also log file(s) which is(are) opened at time.
#
#mix_rbatch_journal_to_logs : true
#mix_rbatch_journal_to_logs : false

# -------------------
# Log setting
# -------------------

# Log Directory
#
#   Default is "<home>/log"
#   <home> is replaced to ${RB_HOME}
#
#log_dir : <home>/rb_log
#log_dir : /var/log/rbatch/

# Log File Name
#
#   Default is "<date>_<time>_<prog>.log".
#   <data> is replaced to YYYYMMDD date string
#   <time> is replaced to HHMMSS time string
#   <prog> is replaced to Program file base name (except extention).
#   <host> is replaced to Hostname.
#
#log_name : "<date>_<time>_<prog>.log"
#log_name : "<date>_<prog>.log"

# Append Log
#
#   Default is ture.
#
#log_append : true
#log_append : false

# Log Level
#
#   Default is "info".
#   Effective values are "debug","info","wran","error",and "fatal".
#
#log_level : "debug"
#log_level : "warn"

# Print log string both log file and STDOUT
#
#   Default is false.
#
#log_stdout : true
#log_stdout : false

# Delete old log files
#
#   Default is false.
#   If this is true, delete old log files when RBatch::Log.new is called.
#   If log filename format does not include "<date>", do nothing.
#
#log_delete_old_log : true
#log_delete_old_log : false

# Expire Days of Log Files
#
#   Default is 7.
#
#log_delete_old_log_date : 14

# Send Mail
#
#   Default is false.
#   When log.error is called, log.fatal is called,
#   or rescue an Exception, send e-mail.
#
#log_send_mail : true

# Mail Parameters
#
#log_mail_to   : "xxx@sample.com"
#log_mail_from : "xxx@sample.com"
#log_mail_server_host : "localhost"
#log_mail_server_port : 25
#
# if you want to send multiple recipients...
#
#log_mail_to :
#  - "AAA@sample.com"
#  - "BBB@sample.com"

# -------------------
# Cmd setting
# -------------------

# Raise Exception
#
#   Default is false.
#   If true, when command exit status is not 0, raise exception.
#
#cmd_raise : true
#cmd_raise : false

# Command Timeout
#
#   Default is 0 [sec] (=no timeout).
#
#cmd_timeout : 5


```

### Customize by passing option object to constructor

If you want to change options in a script, you pass an options object to the constructor of RBatch::Log or RBatch::Cmd.

#### option of RBatch::Log

Sumple

```ruby
opt = {
      :name      => "<date>_<time>_<prog>.log",
      :dir       => "/var/log",
      :append    => true,
      :level     => "info",
      :stdout    => false,
      :delete_old_log => false,
      :delete_old_log_date => 7,
      :send_mail => false,
      :mail_to   => nil,
      :mail_from => "rbatch.localhost",
      :mail_server_host => "localhost",
      :mail_server_port => 25
      }
RBatch::Log.new(opt)
```

#### option of RBatch::Cmd

Sample

```ruby
opt = {
      :raise     => false,
      :timeout   => 0
      }
RBatch::Cmd.new("ls -l", opt).run
```

Migration from version 1 to version 2
--------------

Move `${RB_HOME}/conf/rbatch.yaml` to `${RB_HOME}/.rbatchrc` .
That's all.