Chapter 1
Introduction

Dfect is an assertion testing library for Ruby that emphasizes a simple assertion vocabulary, instant debuggability of failures, and flexibility in composing tests.

What’s new? — history of project releases.
Source code — obtain via Git or browse online.
API reference — documentation for source code.
Project home — the Dfect project home page.

To get help or provide feedback, simply contact the author(s).

Features
Motivation
Etymology
License
Credits
Related works

1.1 Features

Dfect is exciting because:

It has only 5 methods to remember: D F E C T.
It lets you debug assertion failures interactively.
It keeps a detailed report of assertion failures.
It lets you nest tests and execution hooks.
Its core consists of a mere 313 lines of code.

1.2 Motivation

The basic premise of Dfect is that, when a failure occurs, I want to be put inside an interactive debugger where I have the freedom to properly scrutinize the state of my program and determine the root cause of the failure.

Other testing libraries do not fulfill this need. Instead, they simply report each failed assertion along with a stack trace (if I am lucky) and abruptly terminate my program.

This deliberate separation of fault (my program being in an erroneous state) and cause (the source code of my program which caused the fault) reduces me to a primitive and laborious investigative technique known as ”printf debugging”.

If you are not the least bit unsettled by those two words, then recall your first encounter with IRB, the interactive Ruby shell: remember how you would enter code expressions and IRB would instantly evaluate them and show you the result?

What an immense productivity boost! A stark contrast to the endless toil of wrapping every such experiment in standard boilerplate (public static void…), saving the result to a correctly named file, invoking the C/C++/Java compiler, and finally executing the binary—only to be greeted by a segfault. ;-)

I exaggerate, for the sake of entertainment, of course. But my point is that the Ruby testing libraries of today have (thus far) limited our productivity by orphaning us from the nurturing environment of IRB and shooing us off to a barren desert of antiquated techniques. How cruel!

And that, I say, is why Dfect is essential to Ruby developers today. It reunites us with our playful, interactive, real-time IRB roots and, with unwavering tenacity, enables us to investigate failures productively!

1.3 Etymology

Dfect is named after the D F E C T methods it provides.

The name is also play on the word “defect”, whereby the intentional misspelling of “defect” as “dfect” is a defect in itself! ;-)

This wordplay is similar to Mnesia’s play on the word “amnesia”, whereby the intentional omission of the letter “A” indicates forgetfulness—the key characteristic of having amnesia. Clever!

1.4 License

(the ISC license)

Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies.

THE SOFTWARE IS PROVIDED “AS IS” AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.

1.5 Credits

Dfect is made possible by contributions from users like you:

Chapter 2
Setup

Requirements
Installation
Version numbers

2.1 Requirements

Your system needs the following software to run Dfect.

Software	Description	Notes
Ruby	Ruby language interpreter	Version 1.8.6, 1.8.7, and 1.9.1 have been tested successfully.
RubyGems	Ruby packaging system	Version 1.3.1 is or newer required.
ruby-debug	Interactive debugger	This requirement is optional. If this library is not available, then IRB (the standard interactive Ruby shell) will be used instead.

2.2 Installation

You can install Dfect by running this command:

gem install dfect

If you want to develop Dfect, run this command:

gem install dfect --development

2.3 Version numbers

Dfect releases are numbered in major.minor.patch form according to the RubyGems rational versioning policy, which can be summarized as follows:

What increased in the version number?	Is backward compatible?	Has new features?	Has bug fixes?
What increased in the version number?	The increase indicates that the release:
major	No	Yes	Yes
minor	Yes	Yes	Yes
patch	Yes	No	Yes

Chapter 3
Usage

Begin by loading Dfect into your program:

require 'rubygems' # only necessary if you are using Ruby 1.8
require 'dfect'

You now have access to the Dfect module, which provides methods that can be mixed-in or called directly, according to your preference:

Dfect.D "hello" do  # D() is a class method
  puts "world"
end

# the above is same as:

include Dfect       # mix-in the Dfect API

D "hello" do        # D() is an instance method
  puts "world"
end

The following sections explain these provided methods in detail. If you are impatient, you can skip to A sample unit test for an illustrative example.

Assertions
Tests
Execution
Reporting
Emulation

3.1 Assertions

The following methods accept a block parameter and assert something about the result of executing that block. They also accept an optional message, which is shown in failure reports if they fail.

See the API documentation for details and examples.

Method	Description
T	assert true (not `nil` and not `false`)
F	assert not true (`nil` or `false`)
E	assert that an execption is raised
C	assert that a symbol is thrown

Negation
Sampling
Failures

Negation

These methods are the opposite of normal assertions.

Method	Description
T!	same as F
F!	same as T
E!	assert that an exception is not raised
C!	assert that a symbol is not thrown

Sampling

These methods allow you to check the outcome of an assertion without including the assertion in the execution report.

Method	Description
T?	returns `true` if T passes; `false` otherwise
F?	returns `true` if F passes; `false` otherwise
E?	returns `true` if E passes; `false` otherwise
C?	returns `true` if C passes; `false` otherwise

3.1.1 Failures

When an assertion fails, details about the failure will be shown:

- fail: block must yield true (!nil && !false)
  code: |-
    [12..22] in test/simple.rb
       12
       13     D "with more nested tests" do
       14       x = 5
       15
       16       T { x > 2 }   # passes
    => 17       F { x > 2 }   # fails
       18       E { x.hello } # passes
       19     end
       20   end
       21
       22   # equivalent of before(:each) or setup()
  vars:
    x: 5
    y: 83
  call:
  - test/simple.rb:17
  - test/simple.rb:3

You will then be placed into a debugger to investigate the failure if the :debug option is enabled in Dfect.options.

Details about all assertion failures and a trace of all tests executed are stored by Dfect and provided by the Dfect.report method.

3.2 Tests

The D() method defines a new test, which is analagous to the describe() environment provided by BDD frameworks like RSpec.

A test may also contain nested tests.

D "outer test" do
  # assertions and logic here

  D "inner test" do
    # more assertions and logic here
  end
end

Hooks
Insulation

3.2.1 Hooks

The D() method provides several entry points (hooks) into the test execution process:

D "outer test" do
  D .<  { puts "before each nested test" }
  D .>  { puts "after  each nested test" }
  D .<< { puts "before all nested tests" }
  D .>> { puts "after  all nested tests" }

  D "inner test" do
    # assertions and logic here
  end
end

A hook method may be called multiple times. Each call registers additional logic to execute during the hook:

D .< { puts "do something" }
D .< { puts "do something more!" }

3.2.2 Insulation

Use the singleton class of a temporary object to shield your test logic from Ruby’s global environment, the code being tested, and from other tests:

class << Object.new
  # your test logic here
end

Inside this insulated environment, you are free to:

mix-in any modules your test logic needs
define your own constants, methods, and classes

For example:

class << Object.new
  include SomeModule
  extend AnotherModule

  YOUR_CONSTANT = 123

  D "your tests here" do
    # your test logic here

    your_helper_method
  end

  def self.your_helper_method
    # your helper logic here

    helper = YourHelperClass.new
    helper.do_something_helpful

    T { 2 + 2 != 5 }
  end

  class YourHelperClass
    # your helper logic here
  end
end

3.3 Execution

You can configure test execution using:

Dfect.options = your_options_hash

You can execute all tests defined thus far using:

Dfect.run

You can stop this execution at any time using:

Dfect.stop

You can view the results of execution using:

puts Dfect.report.to_yaml

See the API documentation for details and examples.

Automatic test execution

3.3.1 Automatic test execution

require 'rubygems'     # only necessary if you are using Ruby 1.8
require 'dfect/auto'   # <== notice the "auto"

The above code will mix-in the Dfect module into your program and will execute all tests defined by your program before it terminates.

3.4 Reporting

You can insert status messages, which can be arbitrary Ruby objects, into the execution report using the Dfect::S() method.

See the API documentation for details and examples.

3.5 Emulation

Dfect provides emulation layers for several popular testing libraries:

dfect/unit — Test::Unit
dfect/mini — Minitest
dfect/spec — RSpec

Simply require() one of these emulation layers into your test suite and you can write your tests using the familiar syntax of that testing library.

See the API documentation for details and examples.

Example 1. A sample unit test

The following code is from Dfect’s very own test suite.

#--
# Copyright protects this work.
# See LICENSE file for details.
#++

require 'dfect/auto'

D 'T()' do
  T { true   }
  T { !false }
  T { !nil   }

  T { 0 } # zero is true in Ruby! :)
  T { 1 }

  D 'must return block value' do
    inner = rand()
    outer = T { inner }

    T { outer == inner }
  end
end

D 'T!()' do
  T! { !true }
  T! { false }
  T! { nil   }

  D 'must return block value' do
    inner = nil
    outer = T! { inner }

    T { outer == inner }
  end
end

D 'T?()' do
  T { T? { true  } }
  F { T? { false } }
  F { T? { nil   } }

  D 'must not return block value' do
    inner = rand()
    outer = T? { inner }

    F { outer == inner }
    T { outer == true }
  end
end

D 'F() must be same as T!()' do
  T { D.method(:F) == D.method(:T!) }
end

D 'F!() must be same as T()' do
  T { D.method(:F!) == D.method(:T) }
end

D 'F?()' do
  T { T? { true  } }
  F { T? { false } }
  F { T? { nil   } }

  D 'must not return block value' do
    inner = rand()
    outer = F? { inner }

    F { outer == inner }
    T { outer == false }
  end
end

D 'E()' do
  E(SyntaxError) { raise SyntaxError }

  D 'forbids block to not raise anything' do
    F { E? {} }
  end

  D 'forbids block to raise something unexpected' do
    F { E?(ArgumentError) { raise SyntaxError } }
  end

  D 'defaults to StandardError when no kinds specified' do
    E { raise StandardError }
    E { raise }
  end

  D 'does not default to StandardError when kinds are specified' do
    F { E?(SyntaxError) { raise } }
  end

  D 'allows nested rescue' do
    E SyntaxError do
      begin
        raise LoadError
      rescue LoadError
      end

      raise rescue nil

      raise SyntaxError
    end
  end
end

D 'E!()' do
  E!(SyntaxError) { raise ArgumentError }

  D 'allows block to not raise anything' do
    E!(SyntaxError) {}
  end

  D 'allows block to raise something unexpected' do
    T { not E?(ArgumentError) { raise SyntaxError } }
  end

  D 'defaults to StandardError when no kinds specified' do
    E! { raise LoadError }
  end

  D 'does not default to StandardError when kinds are specified' do
    T { not E?(SyntaxError) { raise } }
  end

  D 'allows nested rescue' do
    E! SyntaxError do
      begin
        raise LoadError
      rescue LoadError
      end

      raise rescue nil

      raise ArgumentError
    end
  end
end

D 'C()' do
  C(:foo) { throw :foo }

  D 'forbids block to not throw anything' do
    F { C?(:bar) {} }
  end

  D 'forbids block to throw something unexpected' do
    F { C?(:bar) { throw :foo } }
  end

  D 'allows nested catch' do
    C :foo do
      catch :bar do
        throw :bar
      end

      throw :foo
    end
  end

  D 'returns the value thrown along with symbol' do
    inner = rand()
    outer = C(:foo) { throw :foo, inner }

    T { outer == inner }
  end
end

D 'C!()' do
  C!(:bar) { throw :foo }

  D 'allows block to not throw anything' do
    C!(:bar) {}
  end

  D 'allows block to throw something unexpected' do
    T { not C?(:bar) { throw :foo } }
  end

  D 'allows nested catch' do
    C! :bar do
      catch :moz do
        throw :moz
      end

      throw :foo
    end
  end

  D 'does not return the value thrown along with symbol' do
    inner = rand()
    outer = C!(:foo) { throw :bar, inner }

    F { outer == inner }
    T { outer == nil   }
  end
end

D 'D()' do
  history = []

  D .<< { history << :before_all  }
  D .<  { history << :before_each }
  D .>  { history << :after_each  }
  D .>> { history << :after_all   }

  D 'first nesting' do
    T { history.select {|x| x == :before_all  }.length == 1 }
    T { history.select {|x| x == :before_each }.length == 1 }
    F { history.select {|x| x == :after_each  }.length == 1 }
    T { history.select {|x| x == :after_all   }.length == 0 }
  end

  D 'second nesting' do
    T { history.select {|x| x == :before_all  }.length == 1 }
    T { history.select {|x| x == :before_each }.length == 2 }
    T { history.select {|x| x == :after_each  }.length == 1 }
    T { history.select {|x| x == :after_all   }.length == 0 }
  end

  D 'third nesting' do
    T { history.select {|x| x == :before_all  }.length == 1 }
    T { history.select {|x| x == :before_each }.length == 3 }
    T { history.select {|x| x == :after_each  }.length == 2 }
    T { history.select {|x| x == :after_all   }.length == 0 }
  end

  D 'fourth nesting' do
    D .<< { history << :nested_before_all  }
    D .<  { history << :nested_before_each }
    D .>  { history << :nested_after_each  }
    D .>> { history << :nested_after_all   }

    nested_before_each = 0

    D .< do
      # outer values remain the same for this nesting
      T { history.select {|x| x == :before_all  }.length == 1 }
      T { history.select {|x| x == :before_each }.length == 4 }
      T { history.select {|x| x == :after_each  }.length == 3 }
      T { history.select {|x| x == :after_all   }.length == 0 }

      nested_before_each += 1
      T { history.select {|x| x == :nested_before_each }.length == nested_before_each }
    end

    D 'first double-nesting' do
      T { history.select {|x| x == :nested_before_all  }.length == 1 }
      T { history.select {|x| x == :nested_before_each }.length == 1 }
      F { history.select {|x| x == :nested_after_each  }.length == 1 }
      T { history.select {|x| x == :nested_after_all   }.length == 0 }
    end

    D 'second double-nesting' do
      T { history.select {|x| x == :nested_before_all  }.length == 1 }
      T { history.select {|x| x == :nested_before_each }.length == 2 }
      T { history.select {|x| x == :nested_after_each  }.length == 1 }
      T { history.select {|x| x == :nested_after_all   }.length == 0 }
    end

    D 'third double-nesting' do
      T { history.select {|x| x == :nested_before_all  }.length == 1 }
      T { history.select {|x| x == :nested_before_each }.length == 3 }
      T { history.select {|x| x == :nested_after_each  }.length == 2 }
      T { history.select {|x| x == :nested_after_all   }.length == 0 }
    end
  end
end

D 'D.<() must allow inheritance checking when called without a block' do
  F { D < Kernel }
  F { D < Object }
  F { D < Module }
  T { D.class == Module }

  c = Class.new { include D }
  T { c < D }
end

D 'YAML must be able to serialize a class' do
  T { SyntaxError.to_yaml == "--- SyntaxError\n" }
end

D 'stoping #run' do
  Dfect.stop
  raise 'this must not be reached!'
end

Chapter 4
History

Version 1.1.0 (2009-10-27)
Version 1.0.1 (2009-10-07)
Version 1.0.0 (2009-05-03)
Version 0.1.0 (2009-04-28)
Version 0.0.0 (2009-04-13)

4.1 Version 1.1.0 (2009-10-27)

This release adds a new method for emitting status messages and does some internal housekeeping.

Thank you
New features
Housekeeping

Thank you

Iñaki Baz Castillo used Dfect and suggested new features.

New features

Add Dfect::S() method for adding status messages to the execution report. This feature was requested by Iñaki Baz Castillo.

Housekeeping

Remove unused require of ‘delegate’ standard library in ‘dfect/spec’ RSpec emulation layer.
Mention Emulation layers for popular testing libraries.
Mention that assertions take an optional message parameter.
Replace sample unit test with Dfect test suite.
Upgrade user manual to ERBook 9.0.0.

4.2 Version 1.0.1 (2009-10-07)

This release fixes a bug in the Test::Unit emulation library and revises the user manual.

Bug fixes
Housekeeping

Bug fixes

The parameters for the assert_equal() method in the dfect/unit library were in the wrong order.

Housekeeping

Revise user manual to better fit jQuery UI tabs.
Justify the use of eval() in emulation libraries.
Use simpler Copyright reminder at the top of every file.
Make SLOC count in user manual reflect the core library only.
Mark code spans with {:lang=ruby} instead of HTML <code/> tags.
Open source is for fun, so be nice: speak of “related works” instead of “competitors”.

4.3 Version 1.0.0 (2009-05-03)

This release improves default choices, adds emulation layers to mimic other testing libraries, and fixes some bugs.

Incompatible changes
New features
Bug fixes
Housekeeping

Incompatible changes

The :debug option is now enabled by default and is no longer linked to the value of $DEBUG.
Dfect.run() now appends to previous results by default.

This behavior can be disabled by passing false to the method.

New features

Add emulation layers to mimic other testing libraries:
- dfect/unit — Test::Unit
- dfect/mini — Minitest
- dfect/spec — RSpec

Bug fixes

Do not blindly replace Class#to_yaml; it might be fixed someday.

Housekeeping

Add Motivation section in user manual to promote interactive debugging.
Add brief History of this project’s inception.
Remove redundant assertions for F!() and T!() methods in test suite.
Add copyright notice at the top of every file.

4.4 Version 0.1.0 (2009-04-28)

This release adds new variations to assertion methods, fixes several bugs, and improves test coverage.

Thank you
New features
Bug fixes
Housekeeping

Thank you

François Beausoleil contributed patches for both code and tests! :-)

New features

Added negation (m!) and sampling (m?) variations to assertion methods.

These new methods implement assertion functionality missing so far (previously we could not assert that a given exception was NOT thrown) and thereby allow us to fully test Dfect using itself.
Added documentation on how to insulate tests from the global Ruby namespace.

Bug fixes

The E() method did not consider the case where a block does not raise anything as a failure. —François Beausoleil
When creating a report about an assertion failure, an exception would be thrown if any local variables pointed to an empty array.
The Dfect::<() method broke the inheritance-checking behavior of the < class method.

Added a bypass to the originial behavior so that RCov::XX can properly generate a report about code that uses Dfect.
Added workaround for YAML error when serializing a class object:
```
TypeError: can't dump anonymous class Class
```

Housekeeping

Filled the big holes in test coverage. Everything except the runtime debugging logic is now covered by the unit tests.

4.5 Version 0.0.0 (2009-04-13)

For the longest time, I took Test::Unit and RSpec for granted. They were the epitomy of modern Ruby practice; the insurmountable status quo; immortalized in books, conferences, and blogs alike.

Why would anyone think of using anything remotely different, let alone be foolish enough to write an alternative testing library when these are clearly good enough?

Recent experiments in assertion testing libraries smashed my world view:

The status quo was certainly not “good enough”, as I had so blindly believed all these years. In fact, they were verbose behemoths that chose to encode endless permutations of conjecture into methods.

Empowered by this revelation and inspired by Sean O’Halpin’s musing on alternative names for assertion methods, I rose to challenge the status quo.

And so I present to you the first public release of Dfect.

Resource	Origin	License
	Tango Icon Theme © 2005 Tango Desktop Project	Creative Commons Attribution-ShareAlike 2.5 License Agreement
	MediaWiki Monobook Skin © 2007 MediaWiki contributors	GNU General Public License, version 2

Dfect 1.1.0

Assertion testing library for Ruby

Suraj N. Kurapati

27 October 2009

Contents

Examples

Document

Chapter 1
Introduction

1.1 Features

1.2 Motivation

1.3 Etymology

1.4 License

1.5 Credits

Chapter 2
Setup

2.1 Requirements

2.2 Installation

2.3 Version numbers

Chapter 3
Usage

3.1 Assertions

3.1.1 Failures

3.2 Tests

3.2.1 Hooks

3.2.2 Insulation

3.3 Execution

3.3.1 Automatic test execution

3.4 Reporting

3.5 Emulation

Chapter 4
History

4.1 Version 1.1.0 (2009-10-27)

4.2 Version 1.0.1 (2009-10-07)

4.3 Version 1.0.0 (2009-05-03)

4.4 Version 0.1.0 (2009-04-28)

4.5 Version 0.0.0 (2009-04-13)

Dfect 1.1.0

Assertion testing library for Ruby

Suraj N. Kurapati

27 October 2009

Contents

Examples

Document

Chapter 1 Introduction

1.1 Features

1.2 Motivation

1.3 Etymology

1.4 License

1.5 Credits

1.6 Related works

Chapter 2 Setup

2.1 Requirements

2.2 Installation

2.3 Version numbers

Chapter 3 Usage

3.1 Assertions

3.1.1 Failures

3.2 Tests

3.2.1 Hooks

3.2.2 Insulation

3.3 Execution

3.3.1 Automatic test execution

3.4 Reporting

3.5 Emulation

Chapter 4 History

4.1 Version 1.1.0 (2009-10-27)

4.2 Version 1.0.1 (2009-10-07)

4.3 Version 1.0.0 (2009-05-03)

4.4 Version 0.1.0 (2009-04-28)

4.5 Version 0.0.0 (2009-04-13)

Chapter 1
Introduction

Chapter 2
Setup

Chapter 3
Usage

Chapter 4
History