1. difects(1)
  2. Version 3.0.0
  3. difects(1)

NAME

difects - Assertion testing library for Ruby

DIFECTS is an assertion testing library for Ruby that emphasizes a simple assertion vocabulary, instant debuggability of failures, and flexibility in composing tests.

Features

Resources

Issue tracker (report bugs, request features, get help)

http://github.com/sunaku/difects/issues

Source code (browse online or obtain with Git)

http://github.com/sunaku/difects

API documentation

http://snk.tuxfamily.org/lib/difects/api/

Announcements feed

http://snk.tuxfamily.org/lib/difects/ann.xml

Official website

http://snk.tuxfamily.org/lib/difects/

Setup

Prerequisites:

Installing:

gem install difects

Upgrading:

gem update difects

Removing:

gem uninstall difects

SYNOPSIS

difects [OPTIONS] (FILE|GLOB) ...

DESCRIPTION

Evaluates the given FILEs and also files matching the given GLOB patterns.

The exit status of this command reflects the number of errors and assertion failures up to a maximum of 255 (to avoid 8-bit unsigned integer overflow).

OPTIONS

-d, --debug

Launch interactive debugger upon assertion failures.

-h, --help

Display this manual and exit.

-v, --version

Print version number and exit.

TESTS

The D() method creates a new test, which is analagous to the describe keyword in RSpec and the concept of a "test case" in xUnit. A test may contain nested tests (see Insulation below).

D "outer test" do
  # assertions and logic here

  D "inner test" do
    # more assertions and logic here
  end
end

Hooks

A hook is a scheduled point of entry into the test execution process. The D() method provides several of them:

D "outer test" do
  D .<  { puts "before each nested test" }
  D .>  { puts "after  each nested test" }
  D .<< { puts "before all nested tests" }
  D .>> { puts "after  all nested tests" }

  D "inner test" do
    # assertions and logic here
  end
end

A hook method can be called multiple times. Every call to a hook method schedules additional logic to be executed during the hook:

D .< { puts "do something" }
D .< { puts "do something more!" }

Insulation

The D!() method defines a new test that is explicitly insulated from the tests that contain it and also from the top-level Ruby environment. Root-level calls to the D() method are insulated by default.

Inside an insulated test, you are free to mix-in (using the extend keyword, not the include keyword) any modules your test logic needs and also define your own constants, methods, and classes.

Sharing

S()

Mechanism for sharing code. When called with a block, it shares the given block (under a given identifier) for injection into other tests. When called without a block, it injects a previously shared block (under a given identifier) into the environment where it is called.

S!()

Combination of the two uses of the S() method: it lets you simultaneously share a block of code while injecting it into the environment where that method is called.

S?()

Checks whether any code has been shared under a given identifier.

Information

I()

Mechanism for inserting arbitrary Ruby objects into the test execution report. You can think of this method as a way to inform yourself.

I!()

Starts the interactive debugger at the location where it is called.

Execution

Tests are executed in depth-first search (DFS) order.

You can configure the test execution process using:

DIFECTS.debug = your_choice_here
DIFECTS.quiet = your_choice_here

You can execute all tests defined thus far using:

DIFECTS.start

You can stop the execution at any time using:

DIFECTS.stop

You can view the results of execution using:

puts DIFECTS.trace.to_yaml
puts DIFECTS.stats.to_yaml

You can mix-in the DIFECTS module into your program and execute all tests defined by your program before it terminates by simply adding the following line at the top of your program:

require 'difects/auto'

See the API documentation for more information and examples.

ASSERTIONS

The following methods accept a block parameter and assert something about the result of executing that block. They also accept an optional message, which is shown in failure reports (see Failures below) if they fail.

T()

assert true (not nil and not false)

F()

assert not true (nil or false)

E()

assert that an execption is raised

C()

assert that a symbol is thrown

For the T() and F() methods, you may also pass the condition to be asserted as the first argument (instead of passing it as a block). This might result in a less noisy more pleasing syntax, depending on your taste:

D "Lottery" do
  winning_ticket = rand()

  D "My chances of winning" do
    my_ticket = rand()

    # passing the condition as a block:
    F("I won?!  Dream on.") { my_ticket == winning_ticket }

    # passing the condition as an argument:
    F my_ticket == winning_ticket, "I won?!  Dream on."

  end
end

Negation

The following methods are the opposite of normal assertions.

T!()

same as F()

F!()

same as T()

E!()

assert that an exception is not raised

C!()

assert that a symbol is not thrown

Sampling

The following methods let you check the outcome of an assertion without recording a success or failure in the test execution report.

T?()

returns true if T() passes; false otherwise

F?()

returns true if F() passes; false otherwise

E?()

returns true if E() passes; false otherwise

C?()

returns true if C() passes; false otherwise

Failures

Assertions failures are reported in the following manner:

- fail: block must yield true (!nil && !false)
  call:
  - test/simple.rb:17
  - test/simple.rb:3
  code: |-
    [12..22] in test/simple.rb
       12
       13     D "with more nested tests" do
       14       x = 5
       15
       16       T { x > 2 }   # passes
    => 17       F { x > 2 }   # fails
       18       E { x.hello } # passes
       19     end
       20   end
       21
       22   # equivalent of before(:each) or setup()
  bind: test/simple.rb:17
  vars:
    x: (Fixnum) 5
    y: (Fixnum) 83

Failure reports are composed of the following sections:

fail

Description of the assertion failure.

call

Stack trace leading to the point of failure.

code

Source code surrounding the point of failure.

bind

Location where local variables (in the "vars" section) were extracted.

vars

Local variables visible at the point of failure.

After the failure is reported, you will be placed into a debugger to investigate the failure if the DIFECTS.debug option is enabled.

Assertion failure reports can be accessed at any time within the test execution trace provided by the DIFECTS.trace() method.

EMULATION

DIFECTS can emulate several popular testing libraries:

difects/spec

RSpec emulation layer

difects/unit

Test::Unit emulation layer

difects/mini

Minitest emulation layer

difects/long

Readability emulation layer

Simply require one of these emulation layers into your test suite and you can write your tests using the familiar syntax of the testing library it emulates.

difects/spec

This library emulates RSpec by adding the following methods to the DIFECTS module.

after(what, &block)

Defined at lib/difects/spec.rb:21

before(what, &block)

Defined at lib/difects/spec.rb:10

context(*args, &block)

Defined at lib/difects.rb:1126

describe(*args, &block)

Defined at lib/difects.rb:1126

it(*args, &block)

Defined at lib/difects.rb:1126

difects/unit

This library emulates Test::Unit by adding the following methods to the DIFECTS module.

assert(*args, &block)

Defined at lib/difects.rb:1126

assert_empty(collection, message=nil)

Defined at lib/difects/unit.rb:26

assert_equal(expected, actual, message=nil)

Defined at lib/difects/unit.rb:31

assert_in_delta(expected, actual, delta=nil, message=nil)

Defined at lib/difects/unit.rb:36

assert_in_epsilon(expected, actual, delta=nil, message=nil)

Defined at lib/difects/unit.rb:36

assert_include(item, collection, message=nil)

Defined at lib/difects/unit.rb:47

assert_instance_of(klass, object, message=nil)

Defined at lib/difects/unit.rb:52

assert_kind_of(klass, object, message=nil)

Defined at lib/difects/unit.rb:57

assert_match(pattern, string, message=nil)

Defined at lib/difects/unit.rb:67

assert_nil(object, message=nil)

Defined at lib/difects/unit.rb:62

assert_not(*args, &block)

Defined at lib/difects.rb:1126

assert_not_empty(collection, message=nil)

Defined at lib/difects/unit.rb:26

assert_not_equal(expected, actual, message=nil)

Defined at lib/difects/unit.rb:31

assert_not_in_delta(expected, actual, delta=nil, message=nil)

Defined at lib/difects/unit.rb:36

assert_not_in_epsilon(expected, actual, delta=nil, message=nil)

Defined at lib/difects/unit.rb:36

assert_not_include(item, collection, message=nil)

Defined at lib/difects/unit.rb:47

assert_not_instance_of(klass, object, message=nil)

Defined at lib/difects/unit.rb:52

assert_not_kind_of(klass, object, message=nil)

Defined at lib/difects/unit.rb:57

assert_not_match(pattern, string, message=nil)

Defined at lib/difects/unit.rb:67

assert_not_nil(object, message=nil)

Defined at lib/difects/unit.rb:62

assert_not_operator(object, operator, operand, message=nil)

Defined at lib/difects/unit.rb:77

assert_not_raise(*args, &block)

Defined at lib/difects/unit.rb:82

assert_not_respond_to(object, query, message=nil)

Defined at lib/difects/unit.rb:86

assert_not_same(expected, actual, message=nil)

Defined at lib/difects/unit.rb:72

assert_not_send(object, query, *args)

Defined at lib/difects/unit.rb:95

assert_not_throw(symbol, message=nil, &block)

Defined at lib/difects/unit.rb:91

assert_operator(object, operator, operand, message=nil)

Defined at lib/difects/unit.rb:77

assert_raise(*args, &block)

Defined at lib/difects/unit.rb:82

assert_respond_to(object, query, message=nil)

Defined at lib/difects/unit.rb:86

assert_same(expected, actual, message=nil)

Defined at lib/difects/unit.rb:72

assert_send(object, query, *args)

Defined at lib/difects/unit.rb:95

assert_throw(symbol, message=nil, &block)

Defined at lib/difects/unit.rb:91

setup(*args, &block)

Defined at lib/difects.rb:1126

setup!(*args, &block)

Defined at lib/difects.rb:1126

teardown(*args, &block)

Defined at lib/difects.rb:1126

teardown!(*args, &block)

Defined at lib/difects.rb:1126

test(*args, &block)

Defined at lib/difects.rb:1126

difects/mini

This library emulates Minitest by adding the following methods to the DIFECTS module.

refute(*args, &block)

Defined at lib/difects.rb:1126

refute_empty(collection, message=nil)

Defined at lib/difects/unit.rb:26

refute_equal(expected, actual, message=nil)

Defined at lib/difects/unit.rb:31

refute_in_delta(expected, actual, delta=nil, message=nil)

Defined at lib/difects/unit.rb:36

refute_in_epsilon(expected, actual, delta=nil, message=nil)

Defined at lib/difects/unit.rb:36

refute_include(item, collection, message=nil)

Defined at lib/difects/unit.rb:47

refute_instance_of(klass, object, message=nil)

Defined at lib/difects/unit.rb:52

refute_kind_of(klass, object, message=nil)

Defined at lib/difects/unit.rb:57

refute_match(pattern, string, message=nil)

Defined at lib/difects/unit.rb:67

refute_nil(object, message=nil)

Defined at lib/difects/unit.rb:62

refute_operator(object, operator, operand, message=nil)

Defined at lib/difects/unit.rb:77

refute_raise(*args, &block)

Defined at lib/difects/unit.rb:82

refute_respond_to(object, query, message=nil)

Defined at lib/difects/unit.rb:86

refute_same(expected, actual, message=nil)

Defined at lib/difects/unit.rb:72

refute_send(object, query, *args)

Defined at lib/difects/unit.rb:95

refute_throw(symbol, message=nil, &block)

Defined at lib/difects/unit.rb:91

difects/long

This library emulates Readability by adding the following methods to the DIFECTS module.

Catch(*args, &block)

Defined at lib/difects.rb:1126

Catch!(*args, &block)

Defined at lib/difects.rb:1126

Catch?(*args, &block)

Defined at lib/difects.rb:1126

Describe(*args, &block)

Defined at lib/difects.rb:1126

Describe!(*args, &block)

Defined at lib/difects.rb:1126

Error(*args, &block)

Defined at lib/difects.rb:1126

Error!(*args, &block)

Defined at lib/difects.rb:1126

Error?(*args, &block)

Defined at lib/difects.rb:1126

False(*args, &block)

Defined at lib/difects.rb:1126

False!(*args, &block)

Defined at lib/difects.rb:1126

False?(*args, &block)

Defined at lib/difects.rb:1126

Inform(*args, &block)

Defined at lib/difects.rb:1126

Inform!(*args, &block)

Defined at lib/difects.rb:1126

Share(*args, &block)

Defined at lib/difects.rb:1126

Share!(*args, &block)

Defined at lib/difects.rb:1126

Share?(*args, &block)

Defined at lib/difects.rb:1126

True(*args, &block)

Defined at lib/difects.rb:1126

True!(*args, &block)

Defined at lib/difects.rb:1126

True?(*args, &block)

Defined at lib/difects.rb:1126

EXAMPLES

Begin by loading DIFECTS into your program:

require 'rubygems' # might not be necessary; see HACKING
require 'difects'

You now have access to the DIFECTS module, whose methods can be called directly:

DIFECTS.D "hello" do  # D() is a class method
  puts "world"
end

or mixed-in and called implicitly:

include DIFECTS       # mix-in the DIFECTS module

D "hello" do          # D() is an instance method
  puts "world"
end

according to your preference.

Logging information in the execution report

When the following test is run:

require 'difects/auto'

D 'Wizard' do
  I 'Preparing spell to defeat mortal foes...'
end

D 'Magician' do
  I 'Preparing rabbits to pull from hat...', rand(15)
end

D 'Calculator' do
  I Math::PI, [1, 2, 3, ['a', 'b', 'c']], {:foo => 'bar!'}
end

DIFECTS will output the following:

--- 
- Wizard: 
  - Preparing spell to defeat mortal foes...
- Magician: 
  - Preparing rabbits to pull from hat...
  - 14
- Calculator: 
  - 3.141592653589793
  - - 1
    - 2
    - 3
    - - a
      - b
      - c
  - foo: bar!
--- 
time: 0.002947349

Sharing code between tests

When the following test is run:

require 'difects/auto'

S :knowledge do
  I 'Knowledge is power!'
end

D 'Healer' do
  S :knowledge
end

D 'Warrior' do
  S! :strength do
    I 'Strength is power!'
  end
end

D 'Wizard' do
  S :knowledge
  S :strength
end

D 'King' do
  T { S? :knowledge }
  T { S? :strength }
  F { S? :power }
  I 'Power is power!'
end

DIFECTS will output the following:

--- 
- Healer: 
  - Knowledge is power!
- Warrior: 
  - Strength is power!
- Wizard: 
  - Knowledge is power!
  - Strength is power!
- King: 
  - Power is power!
--- 
pass: 3
time: 0.007106484

Insulated and uninsulated tests

When the following test is run:

require 'difects/auto'

D "a root-level test" do
  @outside = 1
  T { defined? @outside }
  T { @outside == 1 }

  D "an inner, non-insulated test" do
    T { defined? @outside }
    T { @outside == 1 }
  end

  D! "an inner, insulated test" do
    F { defined? @outside }
    F { @outside == 1 }

    @inside = 2
    T { defined? @inside }
    T { @inside == 2 }
  end

  F { defined? @inside }
  F { @inside == 2 }
end

DIFECTS will output the following:

--- 
- a root-level test: 
  - an inner, non-insulated test: 
  - an inner, insulated test: 
--- 
pass: 10
time: 0.007744531

HACKING

This section is meant for people who want to develop DIFECTS's source code.

Prerequisites

Install Ruby libraries necessary for development:

gem install difects --development

Infrastructure

Inochi serves as the project infrastructure for DIFECTS. It handles tasks such as building this help manual and API documentation, and packaging, announcing, and publishing new releases. See its help manual and list of tasks to get started:

inochi --help     # display help manual
inochi --tasks    # list available tasks

$LOAD_PATH setup

Ensure that the lib/ directory is listed in Ruby's $LOAD_PATH before you use any libraries therein or run any executables in the bin/ directory.

This can be achieved by passing an option to Ruby:

ruby -Ilib bin/difects
irb -Ilib -r difects

Or by setting the $RUBYLIB environment variable:

export RUBYLIB=lib   # bash, ksh, zsh
setenv RUBYLIB lib   # csh
set -x RUBYLIB lib   # fish

ruby bin/difects
irb -r difects

Or by installing the ruby-wrapper tool.

RubyGems setup

If you use Ruby 1.8 or older, then ensure that RubyGems is activated before you use any libraries in the lib/ directory or run any executables in the bin/ directory.

This can be achieved by passing an option to Ruby:

ruby -rubygems bin/difects
irb -rubygems -r difects

Or by setting the $RUBYOPT environment variable:

export RUBYOPT=-rubygems   # bash, ksh, zsh
setenv RUBYOPT -rubygems   # csh
set -x RUBYOPT -rubygems   # fish

Running tests

Simply execute the included test runner, which sets up Ruby's $LOAD_PATH for testing, loads the included test/test_helper.rb file, and then evaluates all test/**/*_test.rb files:

test/runner

Its exit status will indicate whether all tests have passed. It may also print additional pass/fail information depending on the testing library used in the test/test_helper.rb file.

Contributing

Fork this project on GitHub (see Resources above) and send a pull request.

VERSIONS

This section contains release notes of current and past releases.

Version 3.0.0 (2010-07-24)

This release renames the project to "DIFECTS", reduces cruft, improves the presentation and debuggability of assertion failures, and revises the manual.

Thank you:

Incompatible changes:

New features:

Bug fixes:

Housekeeping:

Version 2.2.0 (2010-04-28)

This release adds a UNIX manual page and a sub-library for full method names.

New features:

Housekeeping:

Version 2.1.0 (2010-03-31)

This release adds a command-line test runner and performs some minor housekeeping.

New features:

Housekeeping:

Version 2.0.0 (2010-03-21)

This release adds the ability to insulate tests from each other, share code between them, makes the order of parameters consistent in the API, improves user interactivity, fixes some bugs, and revises the user manual.

Incompatible changes:

New features:

Bug fixes:

Housekeeping:

Version 1.1.0 (2009-10-27)

This release adds a new method for emitting status messages and does some internal housekeeping.

Thank you:

New features:

Housekeeping:

Version 1.0.1 (2009-10-07)

This release fixes a bug in the Test::Unit emulation library and revises the user manual.

Bug fixes:

Housekeeping:

Version 1.0.0 (2009-05-03)

This release improves default choices, adds emulation layers to mimic other testing libraries, and fixes some bugs.

Incompatible changes:

New features:

Bug fixes:

Housekeeping:

Version 0.1.0 (2009-04-28)

This release adds new variations to assertion methods, fixes several bugs, and improves test coverage.

Thank you:

New features:

Bug fixes:

Housekeeping:

Version 0.0.0 (2009-04-13)

For the longest time, I took Test::Unit and RSpec for granted. They were the epitomy of modern Ruby practice; the insurmountable status quo; immortalized in books, conferences, and blogs alike.

Why would anyone think of using anything remotely different, let alone be foolish enough to write an alternative testing library when these are clearly good enough?

Recent experiments in assertion testing libraries smashed my world view:

The status quo was certainly not "good enough", as I had so blindly believed all these years. In fact, they were verbose behemoths that chose to encode endless permutations of conjecture into methods.

Empowered by this revelation and inspired by Sean O'Halpin's musing on alternative names for assertion methods, I rose to challenge the status quo.

And so I present to you the first public release of "Dfect".

AUTHORS

Suraj N. Kurapati

CREDITS

François Beausoleil, Gavin Sinclair, Iñaki Baz Castillo, Sean O'Halpin

LICENSE

(the ISC license)

Copyright 2009 Suraj N. Kurapati sunaku@gmail.com

Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies.

THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

SEE ALSO

assert{ 2.0 }, Testy, Verify

  1. July 2010
  2. difects(1)