= sqsrun

A Generics Worker Executor Service for Amazon SQS. It polls SQS queue and run command or script when a message is received. The message is removed only when the command succeeded.


== Architecture

1. sqsrun receives a message from the queue.
2. sqsrun executes a command or runs a script.
   * if the process takes long time, sqsrun extends visibility timeout. this is repeated until it is killed.
   * if the process takes more long time, sqsrun kills the process.
3. if it succeeded, sqsrun removes the message from the queue.
4. if it failed, sqsrun set the visibility timeout to 0 and expect to be retried.


== Usage

  Usage: sqsrun [options] [-- <ARGV-for-exec-or-run>]
      -k, --key-id ID                  AWS Access Key ID
      -s, --secret-key KEY             AWS Secret Access Key
      -q, --queue NAME                 SQS queue name
      -t, --timeout SEC                SQS visibility timeout (default: 30)
          --create                     Create new queue
          --delete                     Delete a queue
          --attr Key=Value             Set attribute
          --push MESSAGE               Push maessage to the queue
          --list                       List queues
          --configure PATH.yaml        Write configuration file
          --exec COMMAND               Execute command
          --run SCRIPT.rb              Run method named 'run' defined in the script
          --env K=V                    Set environment variable
      -e, --extend-timeout SEC         Threashold time before extending visibility timeout (default: timeout * 3/4)
      -x, --kill-timeout SEC           Threashold time before killing process (default: timeout * 10)
      -X, --kill-retry SEC             Threashold time before retrying killing process (default: 60)
      -i, --interval SEC               Polling interval (default: 1)
      -U, --release-on-fail            Releases lock if task failed so that other node can retry immediately
      -d, --daemon PIDFILE             Daemonize (default: foreground)
      -f, --file PATH.yaml             Read configuration file

One of --list, --create, --delete, --attr, --push, --configure, --exec or --run is required. The behavior of the commands is described below:


=== list

Show list of queues. -k and -s options are required.

_Example:_

  $ sqsrun -k KEY_ID -s SEC_KEY --list


=== create

Create new queue. -k, -s and -q options are required.

_Example:_

  $ sqsrun -k KEY_ID -s SEC_KEY --create -q myqueue


=== delete

Delete a queue. -k, -s and -q options are required. Note that messages in the queue will be lost.

_Example:_

  $ sqsrun -k KEY_ID -s SEC_KEY --delete -q myqueue


=== attr

Set a attribute on a queue. -k, -s and -q options are required.

_Example:_

  $ sqsrun -k KEY_ID -s SEC_KEY -q myqueue --attr MessageRetentionPeriod=3600


=== push

Push a message to the queue. -k, -s and -q options are required.

_Example:_

  $ sqsrun -k KEY_ID -s SEC_KEY -q myqueue --push '{"a":"b"}'


=== configure

Write configuration file and exit. Written configuration file can be used with -f option:

_Example:_

  ## create sqsrun.yaml file
  $ sqsrun --configure sqsrun.yaml -k KEY_ID -s SEC_KEY -q myqueue --run myrun.rb -- my run args

  ## run sqsrun using the configuration file
  $ sqsrun -f sqsrun.yaml


=== exec

Execute a command when a message is received. Body of the message is passed to the stdin. The command have to exit with status code 0 when it succeeded.

-k, -s and -q options are required.

_Example:_

  #!/usr/bin/env ruby
  
  require 'json'
  js = JSON.load(STDIN.read)
  puts "received: #{js.inspect}"
  
  # $ sqsrun -k AWS_KEY_ID -s AWS_SEC_KEY -q SQS_NAME --exec ./this_file

When the kill timeout (-x, --klill-timeout) is elapsed, SIGTERM signal will be sent to the child process. The signal will be repeated every few seconds (-X, --kill-retry).


=== run

This is same as 'exec' except that this calls a method named 'run' defined in the file instead of executing the file. Body of the message is passed to the argument. It is assumed it succeeded if the method doesn't any raise errors.

_Example:_

  require 'json'
  
  def run(msg)
    js = JSON.load(msg)
    puts "received: #{js.inspect}"
  end
  
  # optionally you can define terminate method
  # which will be called when kill timeout is elapsed.
  def terminate
  end
  
  # $ sqsrun -k AWS_KEY_ID -s AWS_SEC_KEY -q SQS_NAME --run ./this_file.rb


When the kill timeout (-x, --klill-timeout) is elapsed, terminate() method will be called (if it is defined). It will be repeated every few seconds (-X, --kill-retry).