# rspec-system

`rspec-system` provides a framework for creating system tests using the `rspec` testing library.

The goal here is to provide facilities to aid in the launching of tests nodes, copying of test content to such nodes, and executing commands on such nodes to be tested with standard rspec assertions within the standard rspec test format.

*Note:* This library is fairly alpha at the moment, and the interface may change at without warning. That said, if you're good at ruby and have an opinion, I'd appreciate patches and improvements to move this further torwards stability.

### Gem installation

The intention is that this gem is used within your project as a development library.

Either install `rspec-system` manually with:

    gem install rspec-system

However it is usually recommended to include it in your `Gemfile` and let bundler install it, by adding the following:

    gem 'rspec-system'

Then installing with:

    bundle install

### Writing tests

Start by creating a helper file in `spec/spec_helper_system.rb` containing something like the following:

    require 'rspec-system/spec_helper'

    RSpec.configure do |c|
      c.system_setup_block = proc do
        include RSpecSystem::Helpers
        # Insert some setup tasks here
        system_run('yum install -y ntp')
      end
    end

Create the directory `spec/system` in your project, make sure your unit tests go into `spec/unit` or somesuch so you can isolate them easily during test time. Add files with the spec prefix ie. `mytests_spec.rb` and make sure they always include the line `require 'spec_helper_system'` eg.:

    require 'spec_helper_system'

    describe 'basics' do
      it 'should cat /etc/resolv.conf' do
        system_run('cat /etc/resolv.conf') do |s,o,e|
          o.should =~ /localhost/
        end
      end
    end

Also consult the example in `example` in the source of this library for more details.

For you reference, here are the list of custom rspec configuration items that can be overriden in your `spec_helper_system.rb` file:

* *system_setup_block* - this accepts a proc that is called after node setup, but before every test (ie. before suite). The goal of this option is to provide a good place for node setup independant of tests.
* *system_tmp* - For some of our activity, we require a temporary file area. By default we just a random temporary path, so you normally do not need to set this.

Currently to get the nice formatting rspec-system specific formatter its recommended to use the Rake task, so the following to your `Rakefile`:

    require 'rspec-system/rake_task'

That will setup the rake task `rake spec:system`.

### Creating a nodeset file

A nodeset file outlines all the node configurations for your tests. The concept here is to define one or more 'nodesets' each nodeset containing one or more 'nodes'. Create the file in your projects root directory as `.nodeset.yml`.

    ---
    default_set: 'centos-58-x64'
    sets:
      'centos-58-x64':
        nodes:
          'main.vm':
            prefab: 'centos-58-x64'
      'debian-606-x64':
        nodes:
          'main.vm':
            prefab: 'debian-606-x64'

The file must adhere to the Kwalify schema supplied in `resources/kwalify-schemas/nodeset_schema.yml`.

* `sets`: Each set contains a series of nodes, and is given a unique name. You can create sets with only 1 node if you like.
* `sets -> [setname] -> nodes`: Node definitions for a set. Each node needs a unique name so you can address each one individualy if you like.
* `sets -> [setname] -> nodes -> [name] -> prefab`: This relates to the prefabricated node template you wish to use. Currently this is the only way to launch a node. Look in `resources/prefabs.yml` for more details.
* `default_set`: this is the default set to run if none are provided with `rake spec:system`. This should be the most common platform normally.

### Prefabs

Prefabs are 'pre-rolled' virtual images, for now its the only way to specify a template. In the future we will probably allow you to specify your own prefab file, and override prefab settings in a nodeset file as well.

The current built-in prefabs are defined in `resources/prefabs.yml`. The current set are based on boxes hosted on <http://puppet-vagrant-boxes.puppetlabs.com> as they have been built by myself and are generally trusted and have a reproducable build cycle (they aren't just 'golden images'). In the future I'll probably expand that list, but attempt to stick to boxes that we have control over.

Prefabs are designed to be generic across different hosting environments. For example, you should be able to use a prefab string and launch an EC2 or Vagrant image and find that the images are identical (or as much as possible). The goal should be that local Vagrant users should find their own local tests pass, and when submitting code this should not change for EC2.

For this reason there are various `provider_specific` settings that apply to different provider types. For now though, only `vagrant` specific settings are provided.

`facts` in the prefab are literally dumps of `facter -p` on the host stored in the prefab file so you can look them up without addressing the machine. These are accessed using the `system_node#facts` method on the helper results and can be used in conditional logic during test runs and setup tasks. Not all the facts are supplied, only the more interesting ones.

### Running tests

Run the system tests with:

    rake spec:system

Instead of switches, we use a number of environment variables to modify the behaviour of running tests. This is more inline with the way testing frameworks like Jenkins work, and should be pretty easy for command line users as well:

* *RSPEC_VIRTUAL_ENV* - the type of virtual environment to run (currently `vagrant` is the only option). I'm undecided about this variable, so assume it might change in the future.
* *RSPEC_SET* - the set to use when running tests (defaults to the `default_set` setting in the projects `.nodeset.yml` file). This string must align with the entries under `sets` in your `.nodeset.yml`.

So if you wanted to run an alternate nodeset you could use:

    RSPEC_SET=nodeset2 rake spec:system

In Jenkins you should be able to use RSPEC\_SET in a test matrix, thus obtaining quite a nice integration and visual display of nodesets in Jenkins.

### Plugins to rspec-system

I want to start an eco-system of plugins for rspec-system, but do it in a sane way. Right now I see the following potential plugin types, if you think you can help please do:

* node providers - that is, abstractions around other virtualisation, cloud or system tools. Right now a NodeSet is tied to a virtual type, but I think this isn't granual enough. Some ideas for future providers are:
    * blimpy - for firing up EC2 and OpenStack nodes, useful for Jenkins integration
    * vmware vsphere - for those who have VMWare vSphere deployed already, this would be an awesome bonus.
    * razor - for launching bare metail nodes for testing purposes. Could be really useful to have baremetal tests for software that needs it like `facter`.
    * manual - not everything has to be 'launched' I can see a need for defining a static configuration for older machines that can't be poked and peeked. Of course, we might need to add cleanup tasks for this case.
* helper libraries - libraries that provide test helpers, and setup helpers for testing development on the software in question.
    * distro - helpers that wrap common linux distro tasks, like package installation.
    * puppet - helpers around installing different versions of puppet, PE as well - firing up masters. Perfect for testing modules I think. Puppet could be used a provisioner for setting up tests, as using shell commands for this purpose is a bit rough, not very idepotent and is re-inventing the wheel.
    * mcollective - for launching the basics, activemq, broker clusters. Useful for testing mcollective agents.
    * puppetdb - helpers for setting up puppetdb, probably using the modules we already have.
    * other config management tools - for the purposes of testing modules against them, or using them for test setup provisioners like I've mentioned before with Puppet.
    * others I'm sure ...

These could be shipped as external gems, and plugged in to the rspec-system framework somehow.