Ratchets

Ratchets started life as Reap, a project assistant application born of a complex Rakefile. Rake is the most widely used task tool for Ruby, but over time, small implementations details led Reap to become it's own independent build system. Reap was a great tool, but it was still very much a work in progress... a program in search of it's source.

Then one day, while working on Ratchets' sister project Facets inspiration struck. The key to the "new Reap" is the same as Facets. Just as Facets is a collection of programming tools, so too the "new Reap" would be a collection of project tools. These tools could be called upon by any program, including utilities Ratcehts would provide, but first and foremost it would be a library. And thus Ratchets was born.

Ratchet's features:

• Reuable collection of project management tools.
• Tools can get settings from a central project information store.
• Tools are accessable via a command-line utility.
• Promotes convention over configuration, but still remains flexible.

If you haven't already done so, you'll first need to install Ratchets. The process is straight-forward. Download the package file, decompress it, 'cd' into the package directory and run setup.rb. Eg.

    $ tar -xvzf ratchets-0.7.0.gzip
    $ cd ratchets-0.7.0
    $ sudo ruby setup.rb
  

Alternatively you can install the gem.

    $ gem install ratchets
  

Once installed you can immediately start using Ratchets. Here's the down low.

Rathcets has a command line tool called, simply enough, project. While there are a few other commands you'll eventually want to know about, this is the main facility. You can use project --help to familiarize yourself with some of it's capability.

In general the first argument given to project is either the name of a built-in tool or the name of a user-defined task, followed by any extra information to effect the results of the script/task.

As mentioned at the end of the previous chapter, Ratchets is utilized primarily through the project command-line interface. By supplying this command the name of a "ratchet" with subsequent options we activate various build procedures.

The first "ratchet" we will cover, as it is likely the first anyone will need, is new. The new tool is be used to generate new project scaffolding or add additional parts to an existing project.

Let look at creating a whole new project. To do this, first create a directory for the new project, cd into it, then invoke project new.

    $ mkdir myproject
    $ cd myproject
    $ project new
    Project ready.
  

And just like that, a standard Ruby project layout is created. If you look at the contents of the directory you will see conventional folders like bin/ lib/ and data/. The conventions followed are according to those established by Minero Akoi's setup.rb. If you are relatively new to Ruby it is a good idea to familiarize yourself with this material.

Besides the standard layout, new can also create a subversion layout which includes the associated branches tags trunk tier. Simply specify the option as --svn or --subversion after new.

    $ project new --svn
  

With or without the subversion tier, new also has a website layout option, --web, which will create an additional tier with an index.html file and some other website related structure, placing the source repository in a src subdirectory. This is a great way to layout a project, btw.

The new command can create even better project scaffolding if we first provide some information about our project. To do that we must first create a project information file. The information in this file can then be used by Ratchets to enhance the new project's scaffolding.

    $ mkdir myproject
    $ cd myproject
    $ project new --info
    CREATED: ProjectInfo
    Please edit the file to suit your project.
  

As you can see this creates a project information file called ProjectInfo. Another, and perhaps better way to create a ProjectInfo file is to copy one from some other project and modify it to suit your needs. That makes it easier to learn how to fill them out. But if you don't have that option or you are already familiar with the layout, then you can use the new command create a blank template.

The name of the project information file has some flexibility. Capitalization, for instance, is insignifficant; projectinfo would do just as well. Also a few alternative namings are supported, namely, project.yaml or just PROJECT (again capitalization doesn't matter). For simplicity sake we will refer to this file as the ProjectInfo file throughout the documentation. Just remember that you can substitue any of these other supported names in it's place to suit your personal preference. If you prefer one of the alternate names when creating the file, you can specify it as a parameter of the --info option.

    $ project new --info project.yaml
  

Rather then 'ProjectInfo', the file will be called 'project.yaml'. Ratchets will let you know if you pick a name it does not recognize.

Once you have edited the ProjectInfo file (more on this in the next section), subsequnelty running project new will create the same project layout as before, but it will add enhanced details to further ease the creation of the new project. For instance, the lib directory will already have subdirectory named appropriately and if you use the --web option, the index.html page will be suitably labeled. And so on.

Of course, if you already have a project with which you wish to use Ratchets, rather than create a whole new project layout you will probably just want to add the ProjectInfo file to it. In that case you simply run project new --info. The project information file will be added and the rest of your project will be undisturbed. Running project new on a pre-existing project will have no effect. It will simply report an error that your project already has content.

The project file is of central importance to Ratchets and the project command. The file is a YAML-formatted file storing shared information from which Ratchets' tools gather default information on how to perform their actions. Most subsequent activity will largely depend on the content of this file. So lets now turn our attention squarely to it.

The structure of the ProjectInfo file is fairly self-explanitory. The header is devoted to common information. This is generally followed by deafult tool settings. And lastly a tasks section is used to define user tasks. Each task entry is a YAML map where the key represent the task name followed by a private type (!!) which identifies the tool it invokes. The next line begins the indented attributes the tool needs to do the job. To a detailed list of parameters each tool accepts have a look at the RDoc API.

    --- %YAML:1.0

    title    : Reap
    name     : reap

    version  : 6.0.0
    status   : 'beta'

    author   : Thomas Sawyer
    created  : '2004-04-01'
    email    : transfirz@zmail.com
    homepage : "http://reap.rubyforge.org"

    summary  : A Ruby Project Management Assistant

    description: >
      Reap comprises a set of tasks commonly needed by
      Ruby package developers/deployers, such as testing,
      packaging, releasing, etc. You can also use Reap
      to create your own custom tasks. Reap utilizes a
      YAML configuration file to harvest common project
      information, significantly simplifying these chores.

    rubyforge:
      project  : reap
      username : transami

    revision:
      tool: darcs
      exclude:
        - doc/api

    package:
      executables : [ reap, rubytest ]
      dependencies:
        - [ facets, '> 1.5' ]
      exclude:
        - snip
        - doc/api

    tasks:

      foo: !!ruby
      script: |
        puts "Foo it is!"

  

As you can the top portion is fairly self-explainitory. After that we see entries related to specific Ratchet tools like package. This entry specifies default parameters to use for any subsequent call of the package tool. We will cover this in more detail in the Tool Utilization documention.

Following this is the tasks section with which we can define our own user-defined tasks. Typically these are specializtions of the buil-in tools, but as you can see by our "silly example" arbitary tasks can be written as well. We will cover this in more detail in the Task Management documentation.

When Ratchets searches for a ProjectInfo file it will move up the directory hierarchy from the current working directory until it finds a ProjectInfo file and will assume the location of that file is your project's source directory unless the file itself specifes that another directory is the source root.

Project has one other subcommand that can be used to verify the project information: info. This simply dumps the parsed contents of the ProjectInfo file to stdout.

    $ project info
  

This may seem trivial, but it can be sometimes be useful to quicky insure information is correct and that you are calling project from an appropriate location. [ed- the order of information is arbitrary, so it looks a bit messy. This will be improved in a future release.]

Ratchets is a very versitile application. Ratchets supports a number of techniques for utilizing it's built-in tools and defining new tasks. Depedending on the desired usage, Ratchets can be a build tool library, or taking advantage of it's own system, can be used as a build tool in its own right.

One easily adopted usage of Ratchets is as a build library invoked from Rake. Rake is the prevalent build tool for Ruby, and an excellent one at that. Ratchets tools can be easily called from any application, so calling them from a Rake task is a natural endeavor. Ratchets goes a step further in its support of Rake however by allowing the built-in tools to be setup as Rake tasks automatically. If this is intended usage jump down to Ratchets a la Rake to learn more.

On the other hand, forgoing a separate build tool, tasks can instead be defined as YAML descriptors and invoked via thae project command-line utility. This makes tasks extremely easy to read and write, and allows project information and task definitions to be jointly located but still universally accessible as pure data. We will cover this usage in Describing Tasks via YAML.

The other alternative, which we will discuss last, is for tasks to be defined as stand-alone executables. This approach is in the spirit of Unix --it's favor of many small tools over single monolithic applications. Ratchets provides strong support for this mode of operation, which we have dubbed the Sake Technique. [ed- In fact, it is my prefered usage.]

In any case, no matter which technique is used. The centralized data resource for project information is readily available. This reapability of information, probably more than any other feature, makes Ratchets so effective.

You can define tasks using YAML descriptors. These can be placed directly in the ProjectInfo file unders a tasks: section. Or kept in a separate file called tasks.

    # Example tasks file

    --- %YAML:1.0

    package: !!package
      distribute   : [ tar.bz2, gem, deb ]
      dir: '../package'

    release: !!release
      host       : rubyforge.org
      username   : transami
      project    : reap
      groupid    : 811
      package    : Reap
      dir        : '../DISTRIBUTION'

    publish: !!publish
      target   : rubyforge
      type     : web
      host     : rubyforge.org
      username : transami
      dir      : web

    rdoc: !!rdoc
      dir: 'web/doc/api'
      main: README

    announce: !!announce
      to       : ruby-talk@ruby-lang.org
      from     : &email transfirz@zmail.com
      domain   : unit.rubyforge.org
      server   : smtp.gmail.com
      port     : 587
      account  : *email
      type     : login  # cram_md5, plain
      security : tls    # ~, tls, ssl (not working yet)
      file     : doc/LATEST  # file contains announcement
      slogan   : REAP THE REWARDS!
      links    : []

  

Once you tasks are defined you can see what tasks are ready to run simply by typing project without any arguments into the command line. For example you might see something like:

    ~/myproj$ project
    [from /home/foome/myprojects/foosys]
      announce       Email project announcement.
      doap           Generate DOAP project file.
      extest         Extract unit-tests from script comments.
      info           Display the ProjectInfo file.
      install        Locally install package using setup.rb.
      package        Build distribution packages.
      publish        Publish documents to the web.
      rdoc           Generate API Documentation.
      release        Release distribution files.
      test           Run unit-tests (each in a separate process).
  

To run a task specifiy the name of the task to project, eg. project announce. Task names take precedence over build script names.

Using Ratchets via Rake is staright-forward, but can be approached in either of two ways.

Since Ratchets' project tools are designed as stand-alone reusable modules, one can access them directly. For instance let's define an RDoc task by calling directly on Ratcehts' Doc.rdoc module method.

    require 'ratchets/doc'

    desc 'rdoc the project'

    task :rdoc do
      Ratchets::Doc.rdoc do |r|
        r.title    = "MyApplication"
        r.main     = "README"
        r.template = "html"
        r.options  = ["--all", "--inline-source"]
        r.include  = ["lib/**/*", "bin/*", "[A-Z]*"]
        r.basedir  = "src"
        r.output   = "rdoc"
      end
    end
  

This usage leaves everything up to the the Rake file. Although most of these fields have reasonable defaults. Nonetheless, no information is being provided to the tool via a project information file, becuase we are invoking Ratchet's underlying rdoc tool directly.

Now let's do the same thing, but via the Project class.

    require 'ratchets/project'

    project = Project.new do |info|
      info.title   = "MyApplication"
      info.basedir = "src"
    end


    desc 'rdoc the project'

    task :rdoc do
      project.rdoc do |r|
        r.main     = "README"
        r.template = "html"
        r.options  = ["--all", "--inline-source"]
        r.include  = ["lib/**/*", "bin/*", "[A-Z]*"]
        r.output   = "rdoc"
      end
    end
  

Here we have created a new Project object and have invoked the rdoc tool via it's interface. This automatically incorporates general information about the project of use to the tool --in this case the project's title and it's basedir. The other fields are rdoc specific so they cannot be shared. But we can go a step further and define a set of tool specific defaults for any rdoc task.

    require 'ratchets/project'

    project = Project.new(
      :title   => "MyApplication"
      :basedir => "src"
      :rdoc    => {
        :main     => 'README'
        :template => "html"
        :options  => ["--all", "--inline-source"]
        :include  => ["lib/**/*", "bin/*", "[A-Z]*"]
        :output   => "rdoc"
      }
    )


    desc 'rdoc the project'

    task :rdoc do
      project.rdoc
    end
  

You'll also notice that we are demonstrating Ratchet's versitility in accepting arguments. The Project.new method can take either a hash or a block. In fact, this is a widely used pattern throughout Ratchets.

One final step. It's is likely we don't need to fuss with each and every tool Ratchets provides us. All-in-all we will probably want most, if not all, of them avaialble to us, and since Ratcehts generally provides reasonable defaults for most fields, we will rarely have to explicitly fill out each one. In fact, every field we gave thus far for rdoc, except title and basedir, are the default settings. So to facilitate this, the project class has an autonew method which automatically generates all the tasks for every project tool Ratchets offers.

    require 'ratchets/project'

    project = Project.autonew(
      :title   => "MyApplication"
      :basedir => "src"
    )
  

Now when you invoke Rake -T you will see a good sized list of available tasks.

The techinque as discussed thus far is quite usable, and those heavily favoring pure Rake usage may wish to venture no further than right here. But there are is one final variation that has it's own benefits. Rather then store the project information as Ruby code within one's Rakefile, the information can be placed in a separate ProjectInfo file (something you are already familiar with if you read about Project Generation). To utilize this file, instead of using the new or autonew methods you instead use the load and autoload methods. The upshot is that your typical Rakefile may have little more than these two-lines:

    require 'ratchets/project'
    Project.autoload
  

Sake follows a non-centralized model of design. As such, Sake is essentially a collection of tools that make it easy to create your own sand-alone task scripts. Here's a basic example session of how such tasks are utilized.

    $ cd sake/src

    $ ls
    bin     data  doc  ProjectInfo  setup.rb
    _darcs  demo  lib  script       work

    $ lt script/
    [/home/monkey/code/ruby/sake/src]
    script/package     # Generate distributable packages
    script/version     # Generate VERSION file
    script/release     # Release packages to hosting service
    script/changelog   # Generate ChangeLog file
    script/rdoc        # Generate API documentaiton

    $ script/version -n
    0.0.1 (2006-08-12)
  

Notice the use of lt. That's an included utility that lists executable script in a given directory. It does this by grabbing the first comment line (that starts with #) it finds.

Ruby task scripts are plan ruby scripts. The only thing special about them are:

1. The bang header tells the system to run the script as a "rubytask"
2. The first comment line describes the task (mainly for use by lt).
3. The name of the main method, if used, must be the same as the scripts (minus extension).

Here's your obligatory hello example.

    #! /usr/bin/env rubytask

    # Hello World
    #
    # This is my first sake script!

    puts "Hello, World!"
  

How this example simply prints "Hello, World!" to stdout immediately. That may be fine one-off cases, but because it does not relegate the execution to a method however, it is not as easily reusable. A better way to write this example is:

    #! /usr/bin/env rubytask

    # Hello World
    #
    # This is my first sake script!

    def hello
      puts "Hello, World!"
    end
  

Keep in mind, for this to work, the file itself must be named 'hello' or 'hello.rb' to match the main method's name.

Ratcehts provides a convenient mechanism by which to call other stand-alone tasks simply by calling them as another method.

    #!/usr/bin/env rubytask

    # General preperation
    #
    # This script preforms a number of tasks to prepare for release.

    testrun
    changelog
    version
  

This will call three external scripts in order: test, changelog and version. Sometimes there is method is already defined with the same name as an external task. In those cases you can manually invoke the extrneal method via the #rubytask method. For instance the above can be also written:

    #!/usr/bin/env rubytask

    # General preperation
    #
    # This script preforms a number of tasks to prepare for release.

    system_task 'testrun'
    system_task 'changelog'
    system_task 'version'
  

The two forms are completely equivalent. The former is simply calling #system_task behined the scenes.

The rubytask method restricts the same task from being run twice in the same execution session, so there's not need to worry about cascading task calls. On the other hand if you wish to reuse a task more than once, here's how to write a script to do so.

    #!/usr/bin/env rubytask

    require_local 'hello'

    # Say hello three times

    hello
    hello
    hello
  

The "Sake" technique of creating tasks for a project takes a little more time to setup. But it has many distinct advantages. Among them:

• Tasks are clearly enumerable and can be viewed like any other file-system folder.
• Individual tasks can be quickly edited without having to sort through other task definitions.
• Permissions on tasks can be restricted individually.
• Shell-based tools can coexist along side rubytask tools.

What if I want to run a task script, but I'm currently way down in the project's directory tree. I dont want to cd all the way up or type ../ a bunch of times.

Sake provides a utility called ludo which stands for lookup and do. Just prepend that command to your invocation and it will find the executable and execute it.

      % ludo script/info
  

By the way, the ludo command can be used anywhere you like, it is not dependent on Sake to work. Albeit you should exercise some caution when doing so since ludo actively searches up the directory tree for a script to execute.

By the way, if you'd like to really make sake here's the process:

Good luck with that. ;)

At it's core it is a set of integrated tools designed to simplify the life of Ruby application developers and project managers. The built-in tools cover the full range of common project needs, from setting up a standard project structure to packaging and making announcements. Because of commonality between the tools, Reap utilizes a central YAML configuration file(s) to harvest project information. This significantly simplifies usage.

Custom tasks can also be easily created to suit specific project requirements. In this respect Reap is much like Rake.

Reap differentiates itself from the other build tools in a number of ways. It supports a variety of techinues for utilizing tools and defining tasks. Reap can be used as a Rake-clone. In fact Reap is a nearly 100% compatible replacement for Rake[1]. On the other hand, if you can't pull yourself away from Rake, Reap's tasks can also be used via Rake much like any other set of addon Rake tasks.

Reap tasks can also be defined as standard executables, the ability to use YAML to easily setup built-in tasks, the use of a centralized data resource for project information and it's extensive library of built-in tasks. The built-in tasks are extensive enough that Sake can also be thought of as a complete project managment assitant application.

Revision tools are tied to a particular revision control system like SVN or Darcs. The one exception is the backup tool which simply makes a raw compress backup of your source repository. Because of the tie into the particluar revision system you should make sure the scm: entry is set in your ProjectInfo file, otherwise these tools will not be be available. Vaild settings are cvs, svn, git and darcs.

  name        Name of project/backup
  output      The backup directory (eg. '/var/backup')
  folder      The directory to backup. [.]
  

  output      File to save the log. [doc/ChangeLog]
  style       ChangeLog format. [gnu]
  

  version     Version of release.
  status      Status of release (eg. beta, RC1).
  released    Date of release.
  file        File to save the version stamp.
  

  include    File pattern selecting files to analyze.
               Default pattern is 'lib/**/*'.
  label      Types of notes to find [TODO, FIXME].
  

  include    The files to include in analysis.
               Default pattern is 'lib/**/*'.
  exclude    The files to exclude from analysis.
  

  tests       Test files (eg. test/tc_**/*.rb)

                Defaults to typcial selection.
  libs        List of lookup directories to include in
                load path. './lib' is always included.
  live        Flag to quickly deactive use of local libs.
                Test against installed files instead.
  reqs        List of any files to pre-require.
  

   include       Files to include.
   exclude       Files to exclude from the included.
   file          Save the manifest to this file (otherwise stdout).
   digest        Include optional digest type:
                   md5, sha128 (sha1), sha256, sha512
  

  name       Project name.
  keyfile    Pathname top .pem file of private key.
  include    Glob of files to include.
  exclude    Glob of files to exclude.
  output     Dir to store signiture files [data/{name}/sig].
  

  title      Project title to use in documentation.
  output     Directory to store documentation [doc]
  source     Project location to document [.]
  main       File to use as main page [README]
  template   Which RDoc template to use [html]
  include    Files to include in RDocs
  exclude    Files to exclude from those
  options    Pass-thru extra options to RDoc command
  

  include    Files to include.
  

    title        Project title [title].
    version      Project version [version].
    summary      Brief one-line description [summary].
    description  Long description of project [description].
    homepage     Project homepage web address [homepage].
    links        Array of http links to related sites.
    file         File that contains announcement message.
    memo         Embedded announcement message.
    slogan       Motto for you project.
    email        Send email otherwise just display announcement.
  

    subject      Subject of email message.
    from         Message FROM address [email].
    to           Email address to send announcemnt.
    server       Email server to route message.
    port         Email server's port.
    domain       Email server's domain name.
    account      Email account name.
    login        Login type: plain, cram_md5 or login.
    secure       Uses TLS security, true or false?
  

Inherited settings:

  name            Package name.
  version         Package version.
  architecture    Can be any, i368, i686, ppc, etc.
  executables     Executable files in this distribution.

Tool specific settings:

  output          Directory in which to store distributions.
  include         Files to include in distribution.
  exclude         Files to exclude from those.
  formats         List of distribution formats desired. Eg.
                    tgz, tar.gz, tbz, tar.bz2, zip
                    gem, deb, rpm, pac, etc.
  tier            Version tier the package (for use with Library).
  rules           Specify where files should go in package (see below).

Seetings that can be of use depending on the package format.

  category        Software category.
  dependencies    List of packages this depends.
  recommends      List of packages that can be used with this package.
  replaces        List of packages this one replaces.

RubyGems specific settings:

  autorequire
  platform
  require_paths
  

    folder         Distribution directory
    exclude        Distribution types to exclude
    package        Package name
    date           Date of release (defaults to Time.now)
    processor      Processor/Architecture (Any, i386, PPC, etc.)
    release        Release name (default is version number)
    is_public      Public release?

    changes        Changes given in string form.
    notes          Release notes given in string form.
             -or-
    changelog      Change log file
    notelog        Release notes file
    

   options    Command line options to add to shell command.
   

    # Add tool to a tool module (or add to an existing tool module).

    module MyToolModule

      module_function  # very important!

      def simple( keys=nil, &yld )
        keys = (keys||yld).to_openobject

        message = keys.message
        signed  = keys.signed || '-annonymous'

        puts message + "\n\n" + signed
      end

    end

    # To use the new tool via project, add a tool interface.

    class Project

      tool :simple, 'my custom tool'

      def simple( keys )
        keys = keys.to_h
        keys *= {
          :message => info.message
        }
        keys *= (info[:simple] || {})
        MyToolModule.simple(keys)
      end

    end
  

     message: Hi, how are you?

     tasks:

       simple: !!simple
         signed: Your friend, Tom.
  

     $ project simple