Pattern Matching

As we have seen in episode 261, Ruby has some fairly sophisticated data destructuring capabilities, at least for a dynamic object-oriented language. In that episode, we saw how we could take apart and bind the different parts of a dependency specification in a single statement. The parenthesized expression on the left of the assignment mimics the shape of the data, and Ruby takes apart the data structure and assigns the matching parts of it accordingly.

dep = {"hello" => ["hello.c", "foo.o", "bar.o"]}

a = *dep                        # => [["hello", ["hello.c", "foo.o", "bar.o"]]]

((target, (first_preq, *rest_preqs))) = *dep

target                          # => "hello"
first_preq                      # => "hello.c"
rest_preqs                      # => ["foo.o", "bar.o"]

We have also seen, in various episodes, that Ruby has special tools for matching arbitrary objects against a pattern. The "case equality" operator, or "threequals", lets us apply all kinds of tests to an object. We can test its class, whether it matches a regular expression, whether it is within a given range, and so on.

obj = 23

Integer === obj                 # => true
/foo/   === obj                 # => false
(0...100) === obj               # => true

If you have used any functional programming languages with pattern-matching capabilities, such as Haskell or Elixir, you know that they combine the features of matching and destructuring assignment such that they can both be performed at once. Once you've used a language in the pattern-matching family, you might miss having it in Ruby. I know I do, so I thought it might be fun today to look at how we might add this capability to Ruby.

We will start by defining a placeholder object. A placeholder is a simple creature: it has a context and a name. It also has a custom equivalence operator. This operator behaves a bit peculiarly: it takes the value it is supposed to be matching against, and instead assigns it as the value of this placeholder's name inside the context—which it assumes is a hash-like object. Then it simply returs true, regardless of what the value was. In effect, this is a wildcard object like we saw in episode #215, except it also "captures" values as a side effect.

Next we'll define a MatchContext. It descends from BasicObject, so as to have a minimal set of methods defined. It has an internal hash, called bindings. This hash is specialized: when someone asks it for a key it doesn't have, it will return a placeholder for that key instead.

The class also has a method_missing which simply takes any message sent to the object, and looks up the method's name as a key in the bindings hash.

Placeholder = Struct.new(:bindings, :name) do
  def ==(other)
    bindings[name] = other
    true
  end
end

class MatchContext < BasicObject
  def initialize
    @bindings = ::Hash.new { |hash, key| ::Placeholder.new(hash, key) }
  end

  def method_missing(name, *)
    @bindings[name]
  end
end

Let's play around with these classes a little bit. We'll create a new MatchContext. Then we'll send it some random messages. Each time, it returns a placeholder named for that message.

If we try to match one of these placeholders to an arbitrary value, it succeeds. We are able to do this with the case-equality operator even though we didn't explicitly define it, because case-equality delegates to the double-equals equivalence operator by default.

If we then send the same message as before, we no longer get a placeholder. Instead, we get the value that was "captured" by performing an equality test.

require "./pmatch"

m = MatchContext.new

m.foo                           # => #<struct Placeholder bindings={}, name=:foo>
m.bar                           # => #<struct Placeholder bindings={}, name=:bar>

m.foo === 23                    # => true
m.foo                           # => 23

Let's put these classes to work to do some very basic pattern matching. We'll define an old favorite, a Point struct. We'll instantiate a Point object. Then we'll use our MatchContext and do a pattern match against a Point with placeholder values.

The result of the case equality test is true. And when we examine the placeholders, we can see that they are now bound to the X and Y values of the Point we matched on. In other words, we have successfully checked that that an object is a Point and bound its x and y values to variables, all in one go. Well, OK, not actually to variables, per se, but to something close enough.

require "./pmatch"

Point = Struct.new(:x, :y)

p = Point.new(5, 10)
m = MatchContext.new

Point.new(m.x, m.y) === p       # => true

m.x                             # => 5
m.y                             # => 10

How did this happen? Well, Struct derived objects implement threequals in terms of equivalence. And by default, the equivalence test for a Struct is whether the two objects are the same type and whether their attributes are also equivalent. So comparing one Point to another implicitly delegates to the equivalence operators for the x and y attributes.

So far, our placeholders match anything at all. But we'd like the option to be a little more discerning in our matches. For instance, we'd like to be able to assert that the X and Y values of a Point must be integers (and not nil) for the match to succeed.

To make this possible, we need to flesh out the Placeholder class a little bit. We add a new attribute called guards, which defaults to an empty array. And we overload a method for adding new guards, somewhat arbitrarily choosing the right-shift operator for this purpose. In this method we add the right operand to the guards list, and return self to make further chaining possible.

We then add a guard clause to the definition of the equivalence operator. It will perform case-equality matches of all of the guards against the supplied value, and return false if any of those matches fail.

Placeholder = Struct.new(:bindings, :name) do
  def ==(other)
    return false unless guards.all?{ |g| g === other }
    bindings[name] = other
    true
  end

  def guards
    @guards ||= []
  end

  def >>(guard)
    guards << guard
    self
  end
end

class MatchContext < BasicObject
  def initialize
    @bindings = ::Hash.new { |hash, key| ::Placeholder.new(hash, key) }
  end

  def method_missing(name, *)
    @bindings[name]
  end
end

Now when we match using placeholders, we can annotate the placeholders with extra patterns that the corresponding value must match in order to succeed. In this case, we specify that both attributes of a point must be integers in order to match. This succeeds with the point we've been using. But when we change one of the coordinates to nil, the match no longer returns true.

require "./pmatch2"

Point = Struct.new(:x, :y)

p = Point.new(5, 10)
m = MatchContext.new

Point.new(m.x >> Integer, m.y >> Integer) === p       # => true

m.x                             # => 5
m.y                             # => 10

p = Point.new(5, nil)
m = MatchContext.new

Point.new(m.x >> Integer, m.y >> Integer) === p # => false

Now let's use our placeholders on something slightly more practical. Let's say we have a method, get_account_balance. This method may fail, which we'll simulate in this example by giving it an explicit fail argument. If it succeeds, it returns an account balance as a string. If it fails, it returns an array of two elements: first, a symbol indicating that this is an error. And second, a string explaining the problem. Using different return types to indicate success or failure is a common style in pattern-matching programming languages.

We then open a case statement, with the return value of a call to get_account_balance as the object to switch on. For the first case, we specify a placeholder that is constrained to be a String. In that branch, we print out the account balance. For the next case, we specify a pattern which will match an error return. For the second element in the array, we use another placeholder to capture the error explanation.

If we execute this code, we can see that the case statement matches the success return value, and binds the account balance in the process. If we change the method call to force a failure and run the code again, it matches the error case this time. This time it binds the error info to a pseudo-variable that can be used in the error handling code.

As with the struct, this works because Ruby arrays implement case-equality as an alias for equivalence, and determine equivalence by going over the array members one by one and asking them if they are equivalent to their counterpart in the other array.

require "./pmatch2"

def get_account_balance(fail: false)
  if fail
    [:error, "I literally can't even"]
  else
    "$1234.56"
  end
end

m = MatchContext.new

case get_account_balance(fail: true)
when m.balance >> String
  puts "Balance: #{m.balance}"
when [:error, m.info]
  puts "Error: #{m.info}"
end

# >> Error: I literally can't even

This example really shows off the power of pattern-matching: whereas in normal ruby code we would have had to separately extract the values we were interested in after a case match, here we are able to do a case match and assign variables for later use all at the same time.

And that's plenty for today. Happy hacking!

Noah Gibbs is a Ruby and Rails developer. He's the author of "Rebuilding Rails", a neat little book that helps you understand how Ruby on Rails works by rebuilding pieces of it from scratch. Today Noah steps into the RubyTapas kitchen in order to show us how to discover all kinds of useful information about the methods that are available in a Ruby program. It turns out we can find out quite a bit, just by asking the methods about themselves. I'll let Noah explain. Enjoy!

Show Notes:

• Noah on Twitter: @codefolio
• Noah's Blog
• Rebuilding Rails: Understand Rails by Building a Ruby Web Framework by Noah Gibbs

In Ruby, you can call .methods() on an object to see what other methods you can call on it. For instance:

7.methods

Remember that everything in Ruby is an object, so the number 7 is too. That means you can call .methods() on it.

That output is kind of hard to read. It's in no particular order. So I like to sort them:

7.methods.sort

Sort does the right thing, so that looks better.

This list also doesn't tell you, "what *interesting* methods does this have?" There's a fun old trick for that:

7.methods.sort - Object.methods

You can try to do this with any parent class -- not just Object -- but the obvious approach doesn't work quite right:

7.methods.sort - Fixnum.methods

Seven *is* a Fixnum, so you'd expect that to be empty. But .methods() gets called on the Fixnum object itself -- the class Fixnum, not a specific instance of Fixnum. We want the instance methods that an instance of Fixnum would have.

7.methods.sort - Fixnum.instance_methods

That's better. And we can do it with any parent class we want.

7.methods.sort - Integer.instance_methods

Now that's interesting. Apparently Ruby doesn't define things like multiplication or to_f for the Integer class. You wouldn't directly make an instance of class Integer, but that's still a little quirk that we might care about some day -- for instance, if you inherit one of your own objects from class Integer.

So what else can we do with this?

You can ask, "what was that method name again?"

Here's one I use often. I forget the names of some of the Array and Hash methods, and this trick can find them for me:

[].methods.sort - Object.methods

Remember that you have to use an actual array object -- an instance -- for the first argument. You won't get a useful answer if you say this:

Array.methods.sort - Object.methods

because that's checking the Array class object. Not what you want!

Ruby allows you to define a class across many files, so sometimes it's hard to know what methods there are, or where they're defined. It's nice to be able to get the list of methods for an object. But how can we find out more about a specific method? And especially, how can we find out who defined it?

Imagine that somebody has monkeypatched a class. You know some libraries that do this.

I'll add a very simple file that adds a sum method to your arrays and everything else enumerable. You can find it in the files for this episode. I'll require it here as an example of a mysterious third-party library that monkeypatches standard Ruby classes. If you're following along, put this into the same directory you're working in.

# enumerable_monkeypatch.rb
module Enumerable
  def sum
    inject(&:+)                                # => 6, 1, 2.0, nil
  end
end

Now, let's make sure it works.

require "./enumerable_monkeypatch"
(1..99).sum

Now if we look for this method on Array, we'll find it:

[].methods.include?(:sum)

But where did it come from? Ruby can tell us a lot about a method. So first let's grab that method:

[].method(:sum)

What can we do with a method?

[].method(:sum).methods.sort - Object.methods

Hm. "Owner" looks promising. Let's try that.

[].method(:sum).owner

That told us that it's on Enumerable. But maybe we can do even better.

[].method(:sum).source_location

Oh, hey! There's our source file and line number. And now you can find out where any method was defined. Like the chopsticks say, now you can debug anything!

Oh -- one final bit of trivia before we're done. What if we try it on a function that isn't defined in Ruby? "Plus" on Fixnums is a C method, not a method in a Ruby source file.

7.method(:+)

Ruby says, "not telling." And now I'm done telling for this episode. I'm sure you'll enjoy the next one soon!

Destructuring

We've talked a bit about "destructuring" in the past, in episodes about "splatting" such as #80. Today I want to demonstrate an advanced example of destructuring.

Just to quickly review, in a Ruby context, destructuring assignment is the process of assigning multiple variables at one time, with each bound to a different portion of a data structure. A very simple example is assigning both items in a two-element array to variables at the same time. By using the splat operator, we explicitly tell Ruby to "break up" the array and distribute it across the variables on the left side of the assignment. When we then inspect their values, we can see that a has taken on the first array entry, and b has taken on the second.

arr = [2, 3]
(a, b) = *arr
a # => 2
b # => 3

As we've seen in other episodes such as #81, some of the syntax we've used here can be omitted in this simple case of splatting an array. But for the purpose of today's example I wanted to start out by illustrating the full, canonical syntax for destructuring.

Let us now turn our attention to a slightly more complex data structure. In the Rake build tool, dependencies are represented in the form of single-entry hashes. For instance, here's an example of the dependencies for a C program. The final executable, called hello, depends on a file called hello.c. It also depends on some supporting object files called foo.o and bar.o. If any of these files are updated, the executable needs to be recompiled in order to be current.

dep = {"hello" => ["hello.c", "foo.o", "bar.o"]}

This dependency follows a common convention for build tools: the first file in the dependency list is "special". It is the primary source file for the hello executable, whereas the others are supporting libraries that need to also be linked in at compile time.

In order to break out the component parts of this dependency, we could assign one piece at a time. In order to get the dependency target, we grab the first entry in the Hash. Individual hash entries are represented as two-element arrays, so we then take the first element in the resulting array.

The first prerequisite, then, is the first element of the last element of the first hash entry. And the rest of the prerequisites are the second through last elements of the last element of the first hash entry.

dep = {"hello" => ["hello.c", "foo.o", "bar.o"]}

dep.first                          # => ["hello", ["hello.c", "foo.o", "bar.o"]]
target = dep.first.first        # => "hello"
first_preq = dep.first.last.first # => "hello.c"
rest_preqs = dep.first.last[1..-1] # => ["foo.o", "bar.o"]

This code reads about as well as it sounds to describe it. Which is to say, it's complete gobbledygook. Let's try a different approach.

A Hash can be splatted into an Array. The result is an array of two-element arrays - in this case, just one two-element array. And we know from episode #84 that we can destructure nested arrays so long as we mimic the expected structure using parentheses on the left side of the assignment. So we can build up a destructuring assignment which has named slots for the dependency target, the first prerequisite, and all the remaining prerequisites. For the rest of the prerequisites, we make use of the fact that a variable number of elements can be "slurped" into a single variable by preceding it with a star.

Then, on the right side of the assignment, we splat out the dependency into an array, ready to be destructured.

When we examine the resulting variable assignments, we can see that we successfully captured the target, primary prerequisite, and remaining prerequisites. We did it all in a single assignment. And we were able to do it using a parenthesized form that visually echoes the "shape" of the data we are destructuring.

dep = {"hello" => ["hello.c", "foo.o", "bar.o"]}

a = *dep                        # => [["hello", ["hello.c", "foo.o", "bar.o"]]]

((target, (first_preq, *rest_preqs))) = *dep

target                          # => "hello"
first_preq                      # => "hello.c"
rest_preqs                      # => ["foo.o", "bar.o"]

And that's it for today. Happy hacking!

In today's special guest episode, Tom Stuart shows us the benefits of using enumerators as immutable collections.

I think the first time Tom Stuart blew my mind it was with his article "programming with nothing", which showed how to derive an entire programming system---including numbers and control flow operators---using nothing but Ruby procs taking single arguments. He then went on to write the book "Understanding Computation", which I highly recommend.

Today he's out to blow minds once again, with a fascinating exploration of how we can use Ruby's enumerators to expose and operate on immutable collections. Enjoy!

Show Notes

• [YouTube] Programming with Nothing by Tom Stuart
• Understanding Computation: From Simple Machines to Impossible Programs by Tom Stuart

Immutability is the default in some functional programming languages. But in Ruby most objects are mutable, and that’s by design, because mutability is baked into the way most of us think about object-oriented programming.

Sometimes it’s useful for values to be immutable. Knowing that an object won’t change can make our code easier to reason about, and shared mutable state can be hard to get right in general, even in single-threaded programs.

When our objects are mutable, we always need to be aware of the possibility of them changing. To give one example: when we pass an argument into a method, it can be changed without us realising. Here the string 'cherry' is being modified in-place by the #print_pie method:

def print_pie(filling)
  puts filling << ' pie'
end

fruit = 'cherry'
print_pie(fruit)
fruit # => "cherry pie"

If we want to be sure a value won’t get mutated, we can clone it and pass the clone around instead. Now the original value remains unchanged:

fruit = 'cherry'
cloned_fruit = fruit.clone
print_pie(cloned_fruit)
fruit # => "cherry"

Or we can freeze our value, and then we’ll get an exception if anything tries to modify it:

fruit = 'cherry'
frozen_fruit = fruit.freeze
print_pie(frozen_fruit)
fruit # =>
# ~> -:2:in `print_pie': can't modify frozen String (RuntimeError)
# ~>  from -:7:in `<main>'

In this episode we’re going to try a different way of making objects immutable in Ruby — specifically, a way of making immutable collections.

The simplest way to make an immutable collection is to call #freeze on an existing array. As we just saw, this will disable all the methods that allow the array to be changed:

fruits = %w(apple banana cherry damson elderberry)
fruits.freeze
fruits << 'fig' # =>
# ~> -:3:in `<main>': can't modify frozen Array (RuntimeError)

But there’s another way. We can create a more naturally immutable collection by using an enumerator, because out of the box an enumerator only supports reading from a collection, not modifying it, so we don’t need to disable anything to get the behaviour we want.

One option is to expose an existing mutable collection through an enumerator by using the #to_enum method. The resulting enumerator provides an interface that lets us iterate over the underlying collection but doesn’t give us a way to modify it:

fruits = %w(apple banana cherry damson elderberry).to_enum
fruits.entries # => ["apple", "banana", "cherry", "damson", "elderberry"]
fruits << 'fig' # =>
# ~> -:3:in `<main>': undefined method `<<' for #<Enumerator> (NoMethodError)

Alternatively, we can make an enumerator from scratch by generating its contents with a block. Here we yield a series of strings inside the block, and those strings become the contents of the collection:

fruits = Enumerator.new do |yielder|
  yielder.yield 'apple'
  yielder.yield 'banana'
  yielder.yield 'cherry'
  yielder.yield 'damson'
  yielder.yield 'elderberry'
end

fruits.entries # => ["apple", "banana", "cherry", "damson", "elderberry"]

It’s more obvious why this collection must be immutable. There’s not even any underlying data structure to modify; its contents are being generated on the fly by an unchanging block of Ruby code.

fruits << 'fig'
# ~> -:10:in `<main>': undefined method `<<' for #<Enumerator: #<Enumerator::Generator>:each> (NoMethodError)

So we can base an enumerator on an existing array or create it from scratch with a block. Either way, the structure of the collection can’t be modified through the enumerator, but we can still implement operations that create a new collection by adding, removing, reordering or otherwise changing the contents of this one.

The Enumerator class includes the Enumerable module, so we can do all the usual Enumerable stuff with an immutable collection, like mapping a block over it, filtering it and so on.

fruits.map(&:upcase) # => ["APPLE", "BANANA", "CHERRY", "DAMSON", "ELDERBERRY"]
fruits.select { |fruit| fruit.length == 6 } # => ["banana", "cherry", "damson"]

But note that those operations return a new mutable array, not another immutable collection represented as an enumerator:

fruits.class # => Enumerator
fruits.map(&:upcase).class # => Array
fruits.select { |fruit| fruit.length == 6 }.class # => Array

More generally, we can make a modified copy of an immutable collection by writing a new enumerator that iterates over it and yields different values. Here’s how to add an element:

more_fruits = Enumerator.new do |yielder|
  fruits.each do |fruit|
    yielder.yield fruit
  end

  yielder.yield 'fig'
end

This new enumerator iterates over the old one, yielding every value it finds, and afterwards yields an extra value. As a result, it behaves like the old collection with an extra element appended:

more_fruits.entries # => ["apple", "banana", "cherry", "damson", "elderberry"]
fruits.include?('fig') # => false
more_fruits.include?('fig') # => true

We can use a similar technique to remove an element:

fewer_fruits = Enumerator.new do |yielder|
  fruits.each do |fruit|
    yielder.yield fruit unless fruit == 'cherry'
  end
end

This enumerator iterates over the old one and yields all of its elements except one. So, it behaves like the old collection with an element removed:

fewer_fruits.entries # => ["apple", "banana", "damson", "elderberry"]
fruits.include?('cherry') # => true
fewer_fruits.include?('cherry') # => false

Enumerators are lazy, so this technique also works on infinite collections. For example, here’s an infinite collection of even numbers:

even_numbers = Enumerator.new do |yielder|
  n = 2

  loop do
    yielder.yield n
    n += 2
  end
end

We can take as many elements as we like from this collection, as long as we don’t try to take all of them. The enumerator will generate more elements as they’re needed:

even_numbers.next # => 2
even_numbers.next # => 4
even_numbers.next # => 6
even_numbers.next # => 8
even_numbers.next # => 10

even_numbers.take 10 # => [2, 4, 6, 8, 10, 12, 14, 16, 18, 20]

By making a new enumerator that wraps this collection and yields different values, we can make a modified copy. Here’s a version that includes the unlucky number thirteen at the appropriate position:

even_or_unlucky_numbers = Enumerator.new do |yielder|
  even_numbers.each do |n|
    yielder.yield n
    yielder.yield 13 if n == 12
  end
end

even_or_unlucky_numbers.next # => 2
even_or_unlucky_numbers.next # => 4
even_or_unlucky_numbers.next # => 6
even_or_unlucky_numbers.next # => 8
even_or_unlucky_numbers.next # => 10

even_or_unlucky_numbers.take 10 # => [2, 4, 6, 8, 10, 12, 13, 14, 16, 18]

Another advantage of using enumerators instead of calling #clone or #freeze is that those methods are shallow: they protect the structure of the collection at the top level, but not any of the objects inside it.

So cloning an array doesn’t prevent modification of the string objects it contains, and those objects are shared between the original collection and its clone. Here we can see that the original fruits array ends up containing the string 'pineapple' even though it looks like we’re only modifying the clone:

fruits = %w(apple banana cherry damson elderberry)
cloned_fruits = fruits.clone
cloned_fruits.first.prepend 'pine'
fruits # => ["pineapple", "banana", "cherry", "damson", "elderberry"]

Similarly, freezing the array doesn’t make the strings themselves frozen, so there’s nothing to stop us modifying them in-place:

fruits = %w(apple banana cherry damson elderberry)
frozen_fruits = fruits.freeze
frozen_fruits.first.prepend 'pine'
fruits # => ["pineapple", "banana", "cherry", "damson", "elderberry"]

But an enumerator backed by code instead of a concrete data structure is regenerated every time we iterate over it, so even if its generated contents get mutated, we can arrange for the block to provide fresh copies next time we look. In this case, the string literals inside the block will create new string objects every time they’re evaluated, so even if we pull one out and mutate it, the mutated copy won’t show up next time we iterate over the collection:

fruits = Enumerator.new do |yielder|
  yielder.yield 'apple'
  yielder.yield 'banana'
  yielder.yield 'cherry'
  yielder.yield 'damson'
  yielder.yield 'elderberry'
end

fruits.first.prepend 'pine'
fruits.entries # => ["apple", "banana", "cherry", "damson", "elderberry"]

And that’s how to use enumerators to build immutable collections. Thanks for having me as your guest chef today, and happy hacking!

Advanced Next

In episode #260 we introduced the next block control keyword. We saw how it can be used to skip forward when iterating over lists.

As we ended that episode, we were looking at what happens when we use next in a mapping context. We mapped over a list of strings. When we skipped empty strings using next, this produced nil values in the resulting output array.

objects = ["house", "mouse", "", "mush", "",
           "little old lady whispering 'hush'"]

result = objects.map do |o|
  next if o.empty?
  "Goodnight, #{o}"
end

result
# => ["Goodnight, house",
#     "Goodnight, mouse",
#     nil,
#     "Goodnight, mush",
#     nil,
#     "Goodnight, little old lady whispering 'hush'"]

But what if we instead want to substitute a special string whenever the input is blank? Do we have to switch over to using an if/else statement?

As a matter of fact, we can accomplish this goal with next as well. Let's say we just want to return the string "Goodnight, Moon" wherever the input data is empty. By supplying an argument to next, we accomplish exactly that.

objects = ["house", "mouse", "", "mush", "",
           "little old lady whispering 'hush'"]

result = objects.map do |o|
  next "Goodnight, Moon" if o.empty?
  "Goodnight, #{o}"
end

result.compact
# => ["Goodnight, house",
#     "Goodnight, mouse",
#     "Goodnight, Moon",
#     "Goodnight, mush",
#     "Goodnight, Moon",
#     "Goodnight, little old lady whispering 'hush'"]

What we see here is that the argument to next becomes the return value of the current invocation of a block.

Let's look at a more concrete example of using next with an argument. We have a list of filenames, which we want to winnow down using #select. It's a long list, and it's inefficient to iterate over it multiple times. So we want to get all of our rules for inclusion or exclusion in one place. To do that, we use a series of next statements.

The first statement checks that the filename represents an actual file, rather than a directory or a pipe or some other special entity. Since there is no point performing more checks if it's not a regular file, we skip the remaining checks using next with a false argument. This will cause the block to return false, telling #select that the current filename should not be included in the results.

Next we check that the file is readable by the current user, and skip forwards if it is not.

The next check is a little different. It identifies a file with a name that doesn't match the same naming pattern that all the other files have. We happen to know that we want to include that specific file, so we invoke next with a true argument. This skips all remaining tests and tells #select to go ahead and include this entry in the results.

Next is a test intended to exclude zero-length files. And then there is a final test that includes files matching a particular naming scheme.

Dir["../**/*.mp4"].select { |f|
  next false unless File.file?(f)
  next false unless File.readable?(f)
  next true if f =~ /078b-java-dregs\.mp4/
  next false if File.size(f) == 0
  next true if File.basename(f) =~ /^\d\d\d-/
}

This isn't the only way we could have written this code. We also could have structured it as a single chained boolean expression. But I find that boolean expressions tend to become harder to read the bigger they are, especially when they involve a lot of negation. I like how each line in this block is a self-contained rule which could be safely removed by deleting a single line.

I also like the fact that if we wanted to, we could step through these rules one by one in a debugger. That's not always the case with chained boolean expressions.

So far we've been discussing iteration. It's easy to talk about keywords like next and break and redo as if they are loop control operations. Partly because that is the context that they are commonly found in, and partly because it's easier to understand them by analogy to other languages that have dedicated loop-control operations.

It's important to understand, however, that in Ruby these keywords aren't really loop control operations. They are something more generalized: they are block-control operators. They work anywhere that blocks are passed and invoked, regardless if there is anything like iteration going on.

To hopefully make this principle clear, let's write a little method which accepts a block. It yields to the block twice, logging before and after each yield.

There is no looping or iterating over a list going on here, just two yields. Let's call this method, passing it a block. The block will log, invoke next, and then log again.

Finally, we log the moment the method returns.

When we execute this code, we can see that the block is, in fact, executed twice, just as intended. Each time, it invokes next, which causes control to be passed back to the yieldtwice method before the block can do anything else. The last line of the block is never reached. Again we can see how this behaves like an early return, except for a block instead of a method.

def yieldtwice
  puts "Before first yield"
  yield
  puts "Before second yield"
  yield
  puts "After last yield"
end

yieldtwice do
  puts "About to invoke next"
  next
  puts "Can never get here"
end
puts "After method call"

# >> Before first yield
# >> About to invoke next
# >> Before second yield
# >> About to invoke next
# >> After last yield
# >> After method call

Now let's take one last look at the difference between next and break. Instead of invoking next in the block, we'll invoke break.

def yieldtwice
  puts "Before first yield"
  yield
  puts "Before second yield"
  yield
  puts "After last yield"
end

yieldtwice do
  puts "About to invoke break"
  break
  puts "Can never get here"
end
puts "After method call"

# >> Before first yield
# >> About to invoke break
# >> After method call

This time, we can see that execution only gets as far as the first break before the whole method exits. Unlike next, break doesn't just bring a yield to an early end. It cancels the execution of the whole method that triggered the yield. That is, it forces an early return of the call to yieldtwice.

In working through these two examples, we can begin to see how break and next can effectively function as iteration control operators, because in Ruby iteration is always expressed using blocks. But in fact they don't know anything about loops or iteration; they are all about controlling the execution of blocks and the methods that yield to those blocks.

And that's probably enough to digest in one day. Happy hacking!

Next

In the last episode, #258, I made passing mention of the Ruby next keyword. It occurred to me that not everyone might be familiar with this keyword. Just like redo, it's one of those features where you can spend years writing perfectly good code without knowing about it. But once you do know about it, you can use it to save some effort or make certain idioms more expressive. And even if you use next every day, in the episode after this I'll be covering some advanced uses that might just teach you something new.

At its most basic, next works like the continue loop control keyword found in languages like Java or C. For instance, let's say we have a list of objects we need to say goodnight to. Unfortunately, some blank strings have snuck into the list. As a result, our output is messed up.

objects = ["house", "mouse", "", "mush", "",
           "little old lady whispering 'hush'"]

objects.each do |o|
  puts "Goodnight, #{o}"
end

# >> Goodnight, house
# >> Goodnight, mouse
# >> Goodnight,
# >> Goodnight, mush
# >> Goodnight,
# >> Goodnight, little old lady whispering 'hush'

Let's assume that we can't simply remove the blank strings from the list. One way to deal with them would be to put a conditional statement around the puts line. It will only execute the puts if the string is non-empty.

objects = ["house", "mouse", "", "mush", "",
           "little old lady whispering 'hush'"]

objects.each do |o|
  unless o.empty?
    puts "Goodnight, #{o}"
  end
end

# >> Goodnight, house
# >> Goodnight, mouse
# >> Goodnight, mush
# >> Goodnight, little old lady whispering 'hush'

This works fine. But this style of code always bothers me a little bit. By pushing the actual "meat" of the block down inside a guarding conditional, we've given this junk-handling code greater prominence than I feel it deserves.

If this were a method we were writing, I would prefer to write it like this, with a guard clause which returns early if the input is garbage. Once we are past the guard clause, we can mentally dispense with that case entirely. There is no lingering context to keep us thinking about it.

def say_goodnight_to(object)
  return if object.empty?
  puts "Goodnight, #{object}"
end

Obviously, we can't use return inside an #each loop, because that would abort the iteration and force the whole method to return. Instead, we can use the next keyword to tell Ruby to skip forward to the next iteration.

objects = ["house", "mouse", "", "mush", "",
           "little old lady whispering 'hush'"]

objects.each do |o|
  next if o.empty?
  puts "Goodnight, #{o}"
end

# >> Goodnight, house
# >> Goodnight, mouse
# >> Goodnight, mush
# >> Goodnight, little old lady whispering 'hush'

When we run this version, it leaves out the empty entries in the list. In effect, next gives us a way to add guard clauses for blocks.

You might recall that back in episodes #70 and #71 we introduced the break keyword. If you haven't used the break and next keywords much, you might be a little unclear in how they differ from each other. In order to clarify that difference, let's change our next to a break and run the code again.

objects = ["house", "mouse", "", "mush", "",
           "little old lady whispering 'hush'"]

objects.each do |o|
  break if o.empty?
  puts "Goodnight, #{o}"
end

# >> Goodnight, house
# >> Goodnight, mouse

This time, instead of just skipping the blank entries, the iteration over the array halted completely at the first blank string. That's the difference in a nutshell: next skips to the next iteration, whereas break breaks completely out of the method that the block has been passed into. In this case, that method is #each.

Thus far, we've been coding purely in imperative terms. We're iterating over the elements of an array, printing some of them to standard out, and ignoring any return values.

Let's switch that around now, and look at things from a functional perspective. We'll change the #each to a #map, and assign the result to a variable. For now, we'll get rid of the next. And we'll lose the puts and just return the constructed string from each block invocation.

objects = ["house", "mouse", "", "mush", "",
           "little old lady whispering 'hush'"]

result = objects.map do |o|
  "Goodnight, #{o}"
end

result
# => ["Goodnight, house",
#     "Goodnight, mouse",
#     "Goodnight, ",
#     "Goodnight, mush",
#     "Goodnight, ",
#     "Goodnight, little old lady whispering 'hush'"]

The result is an array of strings, including some junk strings. Let's see what happens when we reintroduce our guard clause using next.

objects = ["house", "mouse", "", "mush", "",
           "little old lady whispering 'hush'"]

result = objects.map do |o|
  next if o.empty?
  "Goodnight, #{o}"
end

result
# => ["Goodnight, house",
#     "Goodnight, mouse",
#     nil,
#     "Goodnight, mush",
#     nil,
#     "Goodnight, little old lady whispering 'hush'"]

This time, where the input data had a blank string, we now have nil values. We now know that the return value of a block invocation which is skipped by next is nil.

This is a handy behavior, since it means we can then use compact to filter out all the nil entries.

objects = ["house", "mouse", "", "mush", "",
           "little old lady whispering 'hush'"]

result = objects.map do |o|
  next if o.empty?
  "Goodnight, #{o}"
end

result.compact
# => ["Goodnight, house",
#     "Goodnight, mouse",
#     "Goodnight, mush",
#     "Goodnight, little old lady whispering 'hush'"]

We are not done. I have a lot more to show you about the next keyword. But in the interests of introducing just one idea at a time, I'm going to end for now and pick up the subject again in the next episode. If everything we've seen so far was old hat to you, be patient: the next installment will dig a lot deeper. Until then, happy hacking!

When I think about regular expressions in Ruby, I think of Nell Shamrell. Nell has put a lot of study into regular expressions—how to write them, how to optimize them, and how they are implemented under the covers. She's given some great talks on this subject. I've put some links in the show notes.

Today, she has agreed to step into the RubyTapas kitchen and give us an introduction on using regex capture groups. If you've ever looked at some of the more advanced regex tricks on this show and felt a little lost, this episode should fill in some of the blanks.

Notes:

• @nellshamrell on Twitter
• Beneath the Surface: Harnessing the power of regular expressions in Ruby (RubyConf 2013)

Today I'd like to talk to you about using regular expressions capture groups in Ruby. Capture groups are a way to capture certain parts of my regular expression's match so I can use them later in my regular expression or later code outside of the regex. Let's say I want to make a regex to scan a string looking for basic urls. Here's an example string.

     "site www.rubytapas.com" 
  

Next, I'm going to create a basic regular expression to find the url in that string. I want this regular expresion to match www followed by a literal dot. Notice that I had to escape the dot using a backslash.This tells the regular expression engine to treat this a literal dot not as a dot metacharacter which has different meaning. Followed by any word character appearing one or more times, followed by another dot, followed by any word character appearing one or more times.

     "site www.rubytapas.com" 
     /www.\.\w+\.\w+/ 
  

Now, this is a somewhat contrived example and there very likely are more efficient regular expressions to match urls, but this one illustrates the points I want to make.

So let's say I want to capture the domain name for use later outside the regular expression. In this string the domain name would be "rubytapas".

     # Domain name: rubytapas 
     "site www.rubytapas.com" 
     /www.\.\w+\.\w+/ 
  

Then I also want to capture the top level domain for use later in the program. In the case of this string, it would be "com".

     # Top Level domain: com # Domain name: rubytapas 
     "site www.rubytapas.com" 
     /www.\.\w+\.\w+/ 
  

To capture that domain name, I'm going to enclose the section of the regex meant to match the domain name in parentheses.

     # Top Level domain: com # Domain name: rubytapas 
     "site www.rubytapas.com" 
     /www.\(.\w+)\.\w+/ 
  

Next I'll do the same thing for the section that's meant to match the top level domain.

     # Top Level domain: com # Domain name: rubytapas 
     "site www.rubytapas.com" 
     /www.\(.\w+)\.(\w+)/ 
  

So let's try running this in Ruby. I'm first going to assign my string to a variable, we'll just call it string.

     # Top Level domain: com # Domain name: rubytapas 
    string = "site www.rubytapas.com"

    /www.\(.\w+)\.(\w+)/
  

Then I'm going to assign my regex to a variable called regex.

     # Top Level domain: com # Domain name: rubytapas 
    string = "site www.rubytapas.com"
    regex = /www.\(.\w+)\.(\w+)/
  

I'm first going to match my regex against this string using the equals sign tilde operator. And I'm going to put my regex on one side of this and my string on the other side. This tells Ruby "look in the string for an of it that matches this regex pattern."

     # Top Level domain: com # Domain name: rubytapas 
    string = "site www.rubytapas.com"
    regex = /www.\(.\w+)\.(\w+)/

    regex =~ string
  

And Ruby's going to return back "5." That "5" means the part of the string that matches the regex begins on the fifth character of the string, the character at index 5.

     # Top Level domain: com # Domain name: rubytapas 
    string = "site www.rubytapas.com"
    regex = /www.\(.\w+)\.(\w+)/

    regex =~ string # => 5
  

Now, knowing where my match began is useful, but Ruby offers a few different ways I can get more information about my match. First, let's say I want to see exactly what my match is. One way to do this in Ruby is to type $~. That returns an instance of Ruby's matchdata class for my match. We'll go a little more into matchdata in just a little bit. Notice that it contains the entire part of the string that matched my regex and the results of the two capture groups.

Now I personally find the $~ to by cryptic and not very readable. Fortunately, Ruby has another way to see what my last match was. And that is through using Regexp - impossible to pronounce but important to know, it's the regular expressions class in Ruby - and I'm going to call last_match on that class. And I get back that same matchdata object for our last match. You can see it also shows the results from my capture groups - those subexpressions within my larger regular expression.

Now what about when I want to look at those capture groups individually and maybe use them later in my code? I can view the first capture group by typing in $1, in that case it returns "rubytapas".

Likewise, I can view the second capture group by typing in $2, which returns "com".

Notice that my first capture group is referenced by one, not by zero. If I were to type in zero, I would get back the name of the program that ran the match.

Along with looking at these capture groups, I can also use them later in the program. Let's try interpolating these two capture groups into a string. In my string I'm going to type in "Domain name: ", then interpolate my first capture group, followed by "Top Level Domain: " then I'll interpolate my second capture group. And this interpolates those two capture groups into my string.

And this interpolates those two capture groups into my string.

Now using numbers like this does work, but again it's somewhat cryptic and a little hard to read. A perhaps clearer way to handle capture groups is through Ruby's matchdata class. Working with capture groups is one of the places the matchdata class is most useful.

So let's create a matchdata object using the match method. I'm going to assign it to a variable called "my_match." And I'm going to call match on my regex and pass it in my string.

And I get back that instance of the matchdata class with the full string and the capture groups.

I can then also access the results of my capture groups similar to how I would access the elements of an array. If I type in my_match[1], I'll get back the result of my first capture group.

Likewise, if I type in my_match[2], I'll get back the result of my second capture group.

Again, note that the first capture group begins at 1, not at 0 like an array. If I were to type in my_match[0], I would get back the entire string that matched the larger regular expression.

And that is an intro to using capture groups in your Ruby regular expressions. Happy hacking!

Let's say we have a list of files we want to download. Each file could be retrieved from any one of three different mirror sites. This is important to know, because the mirrors aren't always reliable. Sometimes they are overloaded with clients, or go down for maintenance. If we get an error response from one mirror, we can try the same request on a different one instead of giving up altogether.

For this example, I've simulated the unreliability of the mirrors using WebMock. Every fourth request to any of the mirror hosts will result in a 502 "Bad Gateway" response.

FILES = %W[file1 file2 file3 file4 file5 file6 file7 file8 file9]

MIRRORS = %W[a.example.org b.example.org c.example.org]

require "net/http"
require "webmock"

include WebMock::API

request_count = 0
err = {status: 502}
ok  = {status: 200, body: "OK!"}
stub_request(:get, /.*\.example\.org/)
  .to_return(->(r){ request_count += 1; request_count % 4 == 0 ? err : ok })

In order to download the files, we can loop over the list using #each. To begin with, we'll just use the first mirror and ignore the others. We'll build a URI by combining the mirror and the filename, and we'll log the URI. Then we'll make an HTTP request.

We then make a decision based on the response status. If it's 200, all is well and we log a successful download. But if it's some other status code, we note the failure and terminate the loop with break.

We run this code, and see that it makes four requests before ending with an error.

require "./setup"

FILES.each do |file|
  mirror = MIRRORS.first
  uri = URI("http://#{mirror}/#{file}")
  puts "Requesting #{uri}"
  result = Net::HTTP.get_response(uri)
  if result.code == "200"
    puts "Success!"
  else
    puts "Error #{result.code}"
    break
  end
end

# >> Requesting http://a.example.org/file1
# >> Success!
# >> Requesting http://a.example.org/file2
# >> Success!
# >> Requesting http://a.example.org/file3
# >> Success!
# >> Requesting http://a.example.org/file4
# >> Error 502

In order to make this code robust in the face of failures, we want it to switch mirrors when there is a network problem. But it's not enough to change mirrors and then go around the loop again, because this would mean skipping the file that failed entirely.

A typical approach to a problem like this would be to add an inner loop which re-tried downloading the same filename with successive mirrors. But in Ruby, we can avoid the need to write a second loop.

Instead, we add just two lines of code. First, when a request fails we shift the mirrors array by one, which has the effect of putting the next mirror at the head of the list. Then, we invoke the redo keyword.

redo is a special block control flow operator in Ruby. It causes execution to be thrown back up to the beginning of the current block. But unlike the next keyword, the block does not advance to the next iteration in a sequence. Instead, it is restarted with the same block argument as the last time around.

We can run the code and see the upshot of this behavior. After four requests, there is an error. But instead of terminating, the request is repeated with the same file but a new mirror. Then there are another four requests, followed by another mirror switch, and so on.

require "./setup"

FILES.each do |file|
  mirror = MIRRORS.first
  uri = URI("http://#{mirror}/#{file}")
  puts "Requesting #{uri}"
  result = Net::HTTP.get_response(uri)
  if result.code == "200"
    puts "Success!"
  else
    puts "Error #{result.code}; switching mirrors"
    MIRRORS.shift
    redo
  end
end

# >> Requesting http://a.example.org/file1
# >> Success!
# >> Requesting http://a.example.org/file2
# >> Success!
# >> Requesting http://a.example.org/file3
# >> Success!
# >> Requesting http://a.example.org/file4
# >> Error 502; switching mirrors
# >> Requesting http://b.example.org/file4
# >> Success!
# >> Requesting http://b.example.org/file5
# >> Success!
# >> Requesting http://b.example.org/file6
# >> Success!
# >> Requesting http://b.example.org/file7
# >> Error 502; switching mirrors
# >> Requesting http://c.example.org/file7
# >> Success!
# >> Requesting http://c.example.org/file8
# >> Success!
# >> Requesting http://c.example.org/file9
# >> Success!

redo is very similar to the retry keyword we discussed in episode #257. The difference is that retry is for exception rescue clauses, and redo is for blocks.

Like with retry, we have to take care with redo that we always move the program state forward in some way before redoing. Otherwise, we risk getting stuck in an infinite loop. In today's example, we're ensuring this by shifting a mirror off of the mirror list before every redo. Eventually, with enough failures this will cause the code to run out of mirrors, a scenario we haven't handled yet.

By using redo, we are able to "try, try again" at a given loop iteration, without adding the code complexity of an inner loop. And that's it for today. Happy hacking!

Today I am very happy to welcome into the RubyTapas kitchen guest chef Peter Cooper. Peter publishes the indispensable Ruby Weekly News, my absolute favorite way to stay up to date on everything happening in the Ruby world. He also publishes many other great newsletters on various programming topics. There are links to all these resources in the show notes.

Today, Peter is going to be sharing with us an introduction to using Ruby's bitwise operators for manipulating numbers at the binary level. This can be a tricky topic to teach and to comprehend, but Peter has come up with some terrific visuals that I think you'll find really make these ideas clear. Enjoy!

Show notes:

• Ruby Weekly
• Cooper Press newsletters
• Peter's Ruby 2.0 walkthrough

Hi, this is Peter Cooper, editor of Ruby Weekly and numerous other email newsletters, you can find me on Twitter at @peterc. It’s an honor to be here with you today.

If you’re here solely for object oriented design, refactoring, design philosophy, or anything like that, you might want to give this video a swerve!

So binary. Here’s a quick refresher.

Normally we represent numbers using the digits 0 through 9, the decimal system, we all know what is.

In the binary system we can also represent numbers, but we have just two digits, 0 and 1. Any number can be represented using 0s and 1s, just as with decimal.

Let’s say we want to represent 0. That’s just 0. Easy.

And 1? 1. Easy.

What about 2? We have no digit two, but the number 2 can be represented as 10. That’s not ten, but one-zero. Instead of having tens, hundreds, thousands, and so forth as each column, we have twos, fours, eights, sixteens, thirty-twoths and so on.

Ruby let’s us work with binary representations and also convert between binary and decimal quite easily. Let’s say we want to know what 42 is in binary. We could work it out, but let’s get Ruby to do the work.

42.to_s(2)

What we do is tell Ruby to convert the decimal 42 to a string that represents the binary version.

It’s 101010 .. nothing suspicious about the meaning of life there then..!

We can also represent numbers directly in binary in Ruby as Avdi showed you in episode 1. We prefix with 0b, like so:

0b101010

Note that we can also convert back from a string representation of binary to decimal with to_s:

“101010”.to_i(2)   # => 42

The argument of 2 is just telling to_i that the representation is in base 2, a synonym for “binary”. Without it, Ruby assumes base 10, decimal, and we’d get this:

“101010”.to_i   # => 101010

Now, Ruby lets us perform special operations upon binary representations that we call “bitwise operations”. Bitwise operations just manipulate things at the level of each bit. What’s a bit, you say?

If you take 101010, each digit there is a bit. Each bit is the smallest unique portion of a computer’s memory, whether that’s regular memory, a register, or whatever.

Bit is a shortened version of “binary digit” by the way, and just to tie together some terminology, eight bits is now typically called a byte, although historically a byte has had no specific length and was simply the smallest addressable unit of memory upon a particular computer architecture.

Let’s start with some Ruby specific methods that aren’t exactly bitwise operations but get us going in the right direction.

Let’s take our 42, and then ask Ruby to tell us what individual bits of it are set to.

100[0] # => 0
100[1] # => 1
100[2] # => 0

Using the square brackets method on a number in Ruby lets us “address” individual bits of that number.

Another thing we can do is see the bit “length” of a number. That is, the minimum number of bits that would be required to represent that number. For example:

42.bit_length # => 6

Whereas, 255, the largest number that can be represented in 8 bits is..

255.bit_length # => 8

Note that just one digit higher, 256, we need 9 bits:

256.bit_length # => 9

As an aside, this can be used as an interesting and quick way to determine if a number is a power of two or not. Take a number x and if x’s bit length is not the same as the bit length of x - 1, it must be a power of 2, as the number of bits needed to represent it has just increased by 1:

x = 256
puts “#{x} is a power of 2!” if x.bit_length != (x-1).bit_length

So, now on to the true, primary bitwise operations. You may have heard of them before, they’re called AND, OR, XOR, and NOT.

The AND operation isn’t the same as the logical and operation you might use on an if statement. Instead, it’s an operation that takes two binary digits or even complete numbers and then compares each bit in each respective position, then only applies a 1 on the output if both respective bits on the input are 1 too. This is best shown visually using what’s called a truth table which shows all combinations of inputs and the outputs they result in.

Or in code, we can demonstrate:

(0b101 & 0b100).to_s(2)  # => “100”

Notice that the AND operator is just a single ampersand, unlike the logical and which is a double ampersand &&.

The OR operation is like the AND operation except the output bit is 1 if either of the input bits is a 1.

Or in code..

(0b101 | 0b110).to_s(2) # => “101"
(0b101 | 0b010).to_s(2) # => “111”

XOR is again a bit like OR but with the proviso that the output bit is only 1 if one and exclusively one of the input bits is a 1. So with OR, if both input bits are 1, the output is 1. But with XOR, the output would be 0 unless a single input bit is 1 and the other is 0.

(0b111 ^ 0b111).to_s(2) # => “0"

NOT is, in theory, the easiest, but in practice is a bit of a pain in Ruby. In theory, if you take a string of bits and flip any 1s to 0s and 0s to 1s, you’re good. The truth table is ridiculously simple.

The problem is that due to how numbers are represented internally in Ruby and other languages, flipping all of the bits has the interesting side effect of making them negative.

(~0b101).to_s(2)  # => “-110”

However, a compounding problem here is that Ruby isn’t really returning the internal representation of the number using to_s, as we can analyse here:

(~0b101)[0]  # => 0
(~0b101)[1]  # => 1
(~0b101)[2]  # => 0

This demonstrates the NOT is actually working properly, but due to the way negative numbers are stored and represented, things get complicated when it comes to rendering decimal equivalents. This could be the topic for an entire other video, however, so we will pause there.

Before I show a quick example of practical uses for these operators, I want to quickly touch on another operation that is commonly considered a bitwise operation, but isn’t the classical sense. It’s called shifting.

Take “101” and let’s “shift” it to the left. We can do this in Ruby with two less than signs.

((0b101) << 1).to_s(2)  # => “1010”

What’s happened is our 101 has shifted one position to the left and a 0 has been placed in the rightmost place. We can then shift is back again, by shifting to the right.

((0b1010) >> 1).to_s(2)  # => “101”

Due to how binary is built around powers of 2, this has the interesting side effect of doubling and halving numbers. Let’s try it on decimal:

24 << 1 # => 48
24 << 2 # => 96 (equivalent of 24 * 2 * 2)
24 << 3 # => 192 (equivalent of 24 * 2 * 2 * 2)
37 >> 1 # => 18

.. because we’re working with binary, we get no decimal places on the last one, we just lop off the odd bit.

Shifting is commonly used at the machine code level to optimise multiplications since shifting bits is a lot quicker than performing true multiplication.

If this intrigues you, you might also look up rotation, which is a bit like shifting, except instead of digits being lost off of either end, they get looped around to the other end of the value. Essentially the bits of a value get rotated rather than just shifted.

So how are bitwise operations useful in Ruby or even programming in general? This is mostly an exercise for you, Google “uses for bit wise operations” and you’ll actually find a lot of stuff, but here’s a quick fly through some ideas.

If you’re doing socket programming or interacting with low level C libraries, you’ll often encounter interesting ways data has been packed using binary. For example, in a single byte, we have 8 bits, but you could represent two 4 bit numbers within that.

Let’s say we have the numbers 6 and 9 and we want to represent those separately within a single byte. We could place one number in the lower 4 bits of the byte, and the other in the higher. But how?

Simply saying x = 9 gets us the number 9 into the lower 4 bits, so that was easy! The 6 will take more work.

So all we do is shift 6 left by 4 bits and add it on!

Now what about extracting both of them? One way is to use what’s called a bitmask. What you do is mark the area you want to extract using one number then AND it with the data to pull out only the marked part.

0b11110000 & 105  # => 96

Then shift that right 4 places:

96 >> 4 # => 6

And similarly for lowest 4 bits:

0b00001111 & 105 # => 9

This sort of stuff is very useful to know when working with things like colour values, IP addresses, file formats, or network packets at a low level.

A similar technique is often used in C to store simple on/off flags compactly. Let’s say we want to represent the flags of a blog post.. things like is it private or not, is it published or not, was it deleted or not?

PRIVATE = 1
PUBLISHED = 2
DELETED = 4

flags = 0

Now, to turn on and off bits, you’d just use OR for turning bits on:

flags |= PRIVATE
flags |= PUBLISHED

And then AND for checking if bits are set, a non-zero value represents true:

flags & PRIVATE # => 1
flags & DELETED # => 0

We could then use XOR to turn OFF a flag:

flags ^= PRIVATE
flags & PRIVATE # => 0
flags & PUBLISHED # => 1

Indeed, XOR actually would toggle the flag on and off if you kept using it.

Now if you thought ActiveRecord’s enums were clever, imagine having this on them!

OK, so this is becoming a feast rather than a tapas, but if you want to keep investigating, bitwise operations are also used in things like:

• compression
• checksums
• hashing
• graphics manipulation (think about doing these operations on colour values, such as overlaying two images on top of each other)
• cryptography (you can XOR values with a key value and toggle them back and forth)
• calculating valid network addresses for a subnet
• swapping two variables without an intermediary

And more. But that’s it, so follow me @peterc, subscribe to Ruby Weekly, and goodbye and goodnight!

In the idealized world of pure programs, operations either work the first time or they don't work at all. Sadly, as soon as we start connecting our software to external services this beautiful dream starts to shatter. Servers go down, networks get overloaded. And sometimes we wind up spending as much time writing code to handle an occasional 502 "Bad Gateway" error as we do on the rest of the code combined.

Let's consider some code that wraps a fictional web service. We've used webmock to simulate an unreliable connection that causes exceptions to be raised the first two times we try to make a request.

require "net/http"
require "webmock"
include WebMock::API

stub_request(:get, "www.example.org")
  .to_raise("Packets devoured by rodents")
  .to_raise("Request saw its shadow")
  .to_return(body: "OK")

As a result, when we call our wrapper method, it fails with an exception.

require "./setup"

def make_request
  result = Net::HTTP.get(URI("http://www.example.org"))
  puts "Success: #{result}"
end

make_request

# ~> StandardError
# ~> Packets devoured by rodents
# ~>
# ~> /home/avdi/.gem/ruby/2.1.2/gems/webmock-1.13.0/lib/webmock/response.rb:68:in `raise_error_if_any'
# ~> /home/avdi/.gem/ruby/2.1.2/gems/webmock-1.13.0/lib/webmock/http_lib_adapters/net_http.rb:173:in `build_net_http_response'
# ~> /home/avdi/.gem/ruby/2.1.2/gems/webmock-1.13.0/lib/webmock/http_lib_adapters/net_http.rb:83:in `request'
# ~> /home/avdi/.rubies/ruby-2.1.2/lib/ruby/2.1.0/net/http.rb:1280:in `request_get'
# ~> /home/avdi/.rubies/ruby-2.1.2/lib/ruby/2.1.0/net/http.rb:480:in `block in get_response'
# ~> /home/avdi/.gem/ruby/2.1.2/gems/webmock-1.13.0/lib/webmock/http_lib_adapters/net_http.rb:123:in `start_without_connect'
# ~> /home/avdi/.gem/ruby/2.1.2/gems/webmock-1.13.0/lib/webmock/http_lib_adapters/net_http.rb:150:in `start'
# ~> /home/avdi/.rubies/ruby-2.1.2/lib/ruby/2.1.0/net/http.rb:583:in `start'
# ~> /home/avdi/.rubies/ruby-2.1.2/lib/ruby/2.1.0/net/http.rb:478:in `get_response'
# ~> /home/avdi/Dropbox/rubytapas/257-retry/make_request.rb:12:in `make_request'
# ~> xmptmp-in7623Pte.rb:3:in `<main>'

Knowing that we are dealing with a flaky service, we'd like to update the #make_request method to try again a few times before giving up. In most programming languages this would involve writing a loop. However, in Ruby we have another option.

In the parameters to the #make_request method, we initialize a count of tries remaining. Then, we add a rescue clause to the method. inside the clause, we first log the error. Then we decrement the tries counter. Then we check to see if there are any tries remaining.

If there are, we invoke the retry keyword. If, however, there are no tries left, we re-raise the current exception.

Let's see what happens when we call this method. This time, we can see that the request failed twice, and succeeded the third time.

require "./setup"

def make_request(tries: 3)
  result = Net::HTTP.get(URI("http://www.example.org"))
  puts "Success: #{result}"
rescue => e
  tries -= 1
  puts "Error: #{e}. #{tries} tries left."
  if tries > 0
    retry
  else
    raise e
  end
end

make_request

# >> Error: Packets devoured by rodents. 2 tries left.
# >> Error: Request saw its shadow. 1 tries left.
# >> Success: OK

So what happened here? By invoking retry, we triggered a feature that is not found in many other languages. retry tells the Ruby VM to back execution up to the beginning of the nearest begin/rescue/end block and try again. Since we used the method-level rescue clause, the effective location of the nearest begin block is the beginning of the current method's code.

There are a couple of points worth noting about this code. First off, it is very important when dealing with retry to remember to create and use a counter of some kind. Likewise, it is vital to remember to decrement the counter before retrying, and to check the state of the counter before retrying. Miss any of these points, and we end up with an infinite loop.

Secondly, note how we made the counter a method parameter instead of instantiating it inside the method body. This isn't just to make it easy to override. Remember what we said earlier: retry starts again from the top of the nearest begin block or, in this case, the current method body. If we had instead made the counter a local variable inside the method body, it would be reinitialized to its original value with every retry. Again, we'd get an infinite loop.

def make_request
  tries = 3 # don't do this
  result = Net::HTTP.get(URI("http://www.example.org"))
  puts "Success: #{result}"
rescue => e
  tries -= 1
  puts "Error: #{e}. #{tries} tries left."
  if tries > 0
    retry
  else
    raise
  end
end

So long as we are careful with our tries counter though, retry gives us an elegant and concise way to re-attempt failed operations.

We've got a little more time, so let's pare this code down to the bare essentials before we wrap up. I wrote out an if/else statement for maximum clarity, but we can write this more concisely. Since, by definition, triggering the retry means that nothing after it will be executed, we can reduce the if statement to a statement modifier. We also don't technically need to supply an argument to raise, since it will implicitly re-reaise the current error if nothing else is given.

require "./setup"

def make_request(tries: 3)
  result = Net::HTTP.get(URI("http://www.example.org"))
  puts "Success: #{result}"
rescue => e
  tries -= 1
  puts "Error: #{e}. #{tries} tries left."
  retry if tries > 0
  raise
end

And that's enough for today. Happy hacking!

At the center of a lot of application programming is workflow. Unlike algorithms or object modeling, workflow is all about lists of tasks that have to be performed one after another in order to achieve some business goal. Usually, the execution of later tasks depends on the outcome of earlier tasks in the list.

For instance, consider this highly simplified workflow. First, log in. Then, make a purchase. Next, collect any special offers related to that purchase. Finally, get a receipt for the purchase.

Unfortunately, any of the steps in this workflow can fail. As we can see if we run the code: currently it doesn't get past trying to make a purchase before terminating with an exception.

$n = 0

def login
  puts "Logging in"
  $n += 1
  return :session123
end

def make_purchase(session)
  puts "Making purchase"
  fail "The API was rude to me" if $n < 2
  $n += 1
  :purchase_record
end

def get_special_offers(purchase_record)
  puts "Getting special offers"
  fail "Special offers server is down."
end

def get_receipt(purchase_record)
  puts "Getting receipt"
  fail "I forgot what I was doing" if $n < 3
  fail "I left it in my other pants" if $n < 4
  $n += 1
  :receipt
end

session = login
purchase_rec = make_purchase(session)
offers = get_special_offers(purchase_rec)
receipt = get_receipt(purchase_rec)

# >> Logging in
# >> Making purchase

# ~> RuntimeError
# ~> The API was rude to me
# ~>
# ~> xmptmp-in25515jcB.rb:11:in `make_purchase'
# ~> xmptmp-in25515jcB.rb:30:in `<main>'

The errors that these particular steps usually encounter tend to be transient errors. That is, they are errors that only happen some of the time, due to network connectivity issues, server outages, or other temporary problems.

There are a lot of potential ways we could change this code to make it more robust. Some possible approaches include building a state machine to represent our workflow. Or creating a general-purpose monadic abstraction for chaining together unreliable actions.

These are legitimate strategies with some good arguments in their favor. But I thought it would be interesting to try and see how we might tackle this with as little change or added ceremony as possible, using just some carefully chosen Ruby features.

In order to make the workflow more robust, we add a new helper method called attempt. It accepts an argument determining how many times a task will be retried, and a block which is the action to be attempted.

We surround each step in the workflow with a call to attempt, specifying different numbers of re-try attempts based on our past experience with these actions and our tolerance for waiting before giving up.

When we run the new code, it gets further. But unfortunately, it still doesn't quite succeed. The method for getting special offers is still failing.

The thing is, getting special offers is strictly optional. This is reflected in the fact that we've only permitted it one attempt. We don't want to waste a lot of time trying to retrieve special offers.

$n = 0

def login
  puts "Logging in"
  $n += 1
  return :session123
end

def make_purchase(session)
  $n += 1
  puts "Making purchase"
  fail "The API was rude to me" if $n < 3
  :purchase_record
end

def get_special_offers(purchase_record)
  $n += 1
  puts "Getting special offers"
  fail "Special offers server is down."
end

def get_receipt(purchase_record)
  $n += 1
  puts "Getting receipt"
  fail "I forgot what I was doing" if $n < 4
  fail "I left it in my other pants" if $n < 5
  :receipt
end

def attempt(times: 1)
  yield
rescue => e
  times -= 1
  retry if times > 0
  raise(e)
end

session      = attempt(times: 1)  {login}
purchase_rec = attempt(times: 3)  {make_purchase(session)}
offers       = attempt(times: 1)  {get_special_offers(purchase_rec)}
receipt      = attempt(times: 10) {get_receipt(purchase_rec)}

# >> Logging in
# >> Making purchase
# >> Making purchase
# >> Getting special offers

# ~> RuntimeError
# ~> Special offers server is down.
# ~>
# ~> xmptmp-in25515NTF.rb:19:in `get_special_offers'
# ~> xmptmp-in25515NTF.rb:40:in `block in <main>'
# ~> xmptmp-in25515NTF.rb:31:in `attempt'
# ~> xmptmp-in25515NTF.rb:40:in `<main>'

What we need is a way to alter the error policy for an individual step in the workflow. In order to do this, we add a new keyword argument on_error to the attempt method. We give it a default which is a lambda that simply re-raises the passed error.

Then we replace the raise in the rescue stanza with a call to the on_error handler, using Ruby's shorthand lambda calling syntax.

Back in our workflow, we change the special offers step. We give it a custom error handler, which simply ignores the passed error and does nothing. When we run the code again, it gets all the way to the end. We have successfully made the special offer step optional.

$n = 0

def login
  puts "Logging in"
  $n += 1
  return :session123
end

def make_purchase(session)
  $n += 1
  puts "Making purchase"
  fail "The API was rude to me" if $n < 3
  :purchase_record
end

def get_special_offers(purchase_record)
  $n += 1
  puts "Getting special offers"
  fail "Special offers server is down."
end

def get_receipt(purchase_record)
  $n += 1
  puts "Getting receipt"
  fail "I forgot what I was doing" if $n < 4
  fail "I left it in my other pants" if $n < 5
  :receipt
end

def attempt(times: 1, on_error: ->(e){raise e})
  yield
rescue => e
  times -= 1
  retry if times > 0
  on_error.(e)
end

session = attempt(times: 1)  {login}
purchase_rec = attempt(times: 3)  {make_purchase(session)}
offers = attempt(times: 1, on_error: ->(_){}) {
  get_special_offers(purchase_rec)
}
receipt = attempt(times: 10) {get_receipt(purchase_rec)}

# >> Logging in
# >> Making purchase
# >> Making purchase
# >> Getting special offers
# >> Getting receipt

Now let's throw a wrench in the works. We don't actually want these steps to raise exceptions when they fail. What we really want to do is collect both the exception and some extra contextual information for diagnostic use.

In order to do this, we add a new variable error_details, which starts out nil. We then make a lambda named capture_error, which will accept an exception as an argument and set the error_details variable to a hash of failure data. In the hash we include the actual exception, the time at which it was raised, and the hostname of the current node. For this last item, we also need to require the socket library.

We then go through each of our steps, except for the optional special offers one, changing the error handler to be our custom lambda.

We add a line to our output, examining the state of the error_details variable. Then we alter one of the workflow steps to force it to always fail.

When we run the code, we can see that rather than raising an exception as it did before, it now saves information into the error_details variable.

$n = 0

def login
  puts "Logging in"
  $n += 1
  return :session123
end

def make_purchase(session)
  $n += 1
  puts "Making purchase"
  fail "The API was rude to me" if $n < 100
  :purchase_record
end

def get_special_offers(purchase_record)
  $n += 1
  puts "Getting special offers"
  fail "Special offers server is down."
end

def get_receipt(purchase_record)
  $n += 1
  puts "Getting receipt"
  fail "I forgot what I was doing" if $n < 4
  fail "I left it in my other pants" if $n < 5
  :receipt
end

def attempt(times: 1, on_error: ->(e){raise e})
  yield
rescue => e
  times -= 1
  retry if times > 0
  on_error.(e)
end

require "socket"
error_details = nil
capture_error = ->(e){
  error_details = {
    error: e,
    time:  Time.now,
    host: Socket.gethostname
  }
}
session = attempt(times: 1, on_error: capture_error)  {login}
purchase_rec = attempt(times: 3, on_error: capture_error) {
  make_purchase(session)
}
offers = attempt(times: 1, on_error: ->(_){}) {
  get_special_offers(purchase_rec)
}
receipt = attempt(times: 10, on_error: capture_error) {
  get_receipt(purchase_rec)
}

error_details
# => {:error=>#<RuntimeError: The API was rude to me>,
# :time=>2014-10-17 00:35:15 -0400,
# :host=>"hazel"}

# >> Logging in
# >> Making purchase
# >> Making purchase
# >> Making purchase
# >> Getting special offers
# >> Getting receipt

Unfortunately, we can also see an unintended consequence in the output. Because we are no longer terminating execution early with an exception, the steps go right on executing even after the purchase step has failed.

In order to ensure execution stops as soon as a required step fails, we make a few more changes. First, we modify our error handler lambdas. We make the capture_error one return false, and the special offer's no-op one returns true.

Then we add and operators connecting each of the steps together in a chain. You might recall this idiom from episode #125.

We've now made the execution of each step depend on the previous step returning a truthy value. We happen to know that all of our steps return something truthy when they succeed. And we've ensured that the capture_error lambda returns false, which should end the chain of execution when an error is encountered.

To test this, we run the code again. Sure enough, we can see that the workflow now only gets as far as the failing step, and then stops.

$n = 0

def login
  puts "Logging in"
  $n += 1
  return :session123
end

def make_purchase(session)
  $n += 1
  puts "Making purchase"
  fail "The API was rude to me" if $n < 100
  :purchase_record
end

def get_special_offers(purchase_record)
  $n += 1
  puts "Getting special offers"
  fail "Special offers server is down."
end

def get_receipt(purchase_record)
  $n += 1
  puts "Getting receipt"
  fail "I forgot what I was doing" if $n < 4
  fail "I left it in my other pants" if $n < 5
  :receipt
end

def attempt(times: 1, on_error: ->(e){raise e})
  yield
rescue => e
  times -= 1
  retry if times > 0
  on_error.(e)
end

require "socket"
error_details = nil
capture_error = ->(e){
  error_details = {
    error: e,
    time:  Time.now,
    host:  Socket.gethostname
  }
  false
}
session = attempt(times: 1, on_error: capture_error)  {login} and
purchase_rec = attempt(times: 3, on_error: capture_error) {
  make_purchase(session)
} and
offers = attempt(times: 1, on_error: ->(_){true}) {
  get_special_offers(purchase_rec)
} and
receipt = attempt(times: 10, on_error: capture_error) {
  get_receipt(purchase_rec)
}

error_details
# => {:error=>#<RuntimeError: The API was rude to me>,
# :time=>2014-10-17 00:34:58 -0400,
# :host=>"hazel"}

# >> Logging in
# >> Making purchase
# >> Making purchase
# >> Making purchase

There is a lot more we could try and tackle here. For instance, what about specifying timeouts for slow operations? And what do we do about tasks that return a falsy value even when they succeed?

There are clearly refinements we could make. But what we have already done works well, and I like that we've managed to accomplish without radically changing the basic structure of this code. I think this is enough for today. Happy hacking!

I want to share with you one of my favorite Ruby parlor tricks.

The other day I was updating an old website, and I needed to make sure the changes I had made looked right. It was a purely static site, so I just needed something that would serve the files in the current directory to a browser.

In order to do this, I used the following command line:

ruby -run -e httpd . --port=8080

This booted up a Webrick instance, and I was able to test the site.

Let's talk about what's going on here. We'll start by breaking down the command I used.

The ruby part is hopefully self-explanatory. Then comes a dash and an 'r'. This is where things get a little sneaky. '-r' is mnemonic for require. This flag causes Ruby to treat the next argument as the name of a feature and attempt to load it.

Next we supply the library to be required, which is called simply un. Ruby does not require us to put a space between short-form flags and their arguments, so we can squash them together. The result is that it looks like the word run… which is exactly what the authors playfully intended when they named the library.

The next flag is -e, which tells Ruby to execute the following argument as Ruby code. The code we execute is simply a one-word method invocation: httpd.

Next we supply the current directory as the directory to serve, and specify a port number.

ruby -r un -e httpd . --port=8080

So what is this mysterious un library, and this httpd method?

Let's ask Ruby. As we learned in episode #237, Ruby keeps a list of the library files it has already loaded. We can tell Ruby to load the un library, and then grep through the $LOADED_FEATURES list until we find one that ends with un.rb.

$ ruby -run -e 'puts $LOADED_FEATURES.grep(/un.rb$/)'
/home/avdi/.rubies/ruby-2.1.2/lib/ruby/2.1.0/un.rb

Now that we know where to find this library, lets open it up and take a peek inside.

What we find is a is one-file library that offers a number of handy command-line utilities written in pure Ruby. A lot of them we will probably never need, like clones of the UNIX copy and move commands. Although, these might just be useful if we ever wanted to write a simple script that worked the same on Windows as on Linux or Mac OS.

If we scroll down, we can find the httpd method. It's pretty short and simple. There's come boilerplate for handling command-line flags, and then just enough code to start up a Webrick server, serving the given directory.

And that's really Ruby in a nutshell: it has fun magic tricks, but it's easy to look behind the curtain and learn how to the trick was accomplished. Happy hacking!

So far on this show we have covered a few different ways to iterate over a sequence of incrementing or decrementing integers, without resorting to a traditional for-loop. We've used Ruby ranges, as well as the #upto and #downto= methods. We saw how both of those approaches have their own limitations.

Today, we'll look at a generalization of this concept.

Let's jump back to some code we saw in the last episode, #253. It prints a few verses of the song "99 bottles of beer on the wall.

99.downto(95) do |n|
  puts "#{n} bottles of beer on the wall, #{n} bottles of beer"
  puts "Take one down, pass it around, #{n-1} bottles of beer on the wall"
end

# >> 99 bottles of beer on the wall, 99 bottles of beer
# >> Take one down, pass it around, 98 bottles of beer on the wall
# >> 98 bottles of beer on the wall, 98 bottles of beer
# >> Take one down, pass it around, 97 bottles of beer on the wall
# >> 97 bottles of beer on the wall, 97 bottles of beer
# >> Take one down, pass it around, 96 bottles of beer on the wall
# >> 96 bottles of beer on the wall, 96 bottles of beer
# >> Take one down, pass it around, 95 bottles of beer on the wall
# >> 95 bottles of beer on the wall, 95 bottles of beer
# >> Take one down, pass it around, 94 bottles of beer on the wall

Now let's say we want to get to the end of the song faster, by only printing every other verse. We can't do this with the #downto method. But we can do it with the #step method. We give #step two optional keyword arguments: the number to end on, and the number to add on each iteration. In our case, we tell it to end on 94, and step by -2. The result is an abbreviated rendition of the song.

99.step(to: 95, by: -2) do |n|
  puts "#{n} bottles of beer on the wall, #{n} bottles of beer"
  puts "Take one down, pass it around, #{n-1} bottles of beer on the wall"
end

# >> 99 bottles of beer on the wall, 99 bottles of beer
# >> Take one down, pass it around, 98 bottles of beer on the wall
# >> 97 bottles of beer on the wall, 97 bottles of beer
# >> Take one down, pass it around, 96 bottles of beer on the wall
# >> 95 bottles of beer on the wall, 95 bottles of beer
# >> Take one down, pass it around, 94 bottles of beer on the wall

As I mentioned, both arguments are optional. If we omit the by keyword, we get an upwards count by one.

1.step(to: 5) do |n|
  puts n
end

# >> 1
# >> 2
# >> 3
# >> 4
# >> 5

If we omit the to keyword, we get an infinite progression. Of course, this means we need to have some condition for breaking out of the block unless we want it to loop infinitely. Here's an example of a loop that searches for an unused ID.

ids = [3, 2, 5, 4, 1, 7]
id = 1.step do |n|
  break n if !ids.include?(n)
end
puts "Free ID: #{id}"

# >> Free ID: 6

(If the idiom of using break to return a value from a block is unfamiliar to you, check out episode #71.)

Also unlike the #upto and #downto methods, #step is available on all Numeric types, not just integers. This means that we could increment floating point numbers if we wanted to.

0.0.step(to: 1.0, by: 0.05) do |n|
  puts n
end

# >> 0.0
# >> 0.05
# >> 0.1
# >> 0.15000000000000002
# >> 0.2
# >> 0.25
# >> 0.30000000000000004
# >> 0.35000000000000003
# >> 0.4
# >> 0.45
# >> 0.5
# >> 0.55
# >> 0.6000000000000001
# >> 0.65
# >> 0.7000000000000001
# >> 0.75
# >> 0.8
# >> 0.8500000000000001
# >> 0.9
# >> 0.9500000000000001
# >> 1.0

Finally, as you might imagine from some of the other iteration methods we've explored on this show, there is also a form of #step that returns an Enumerator. Want to know the sum of just the even numbers from 0 through 100? We can set up the iteration and then, instead of supplying a block directly to the #step message, we can chain on sends to other Enumerable methods. In this case, we use #reduce to calculate a sum.

0.step(to: 100, by: 2).reduce(:+) # => 2550

(We introduced this usage of #reduce in episode #149)

While it isn't quite as expressive as special case methods like #upto and #downto, #step enables us to iterate over numeric sequences in a very generalized and flexible way. Hopefully, knowing about #step will save you from having to write a specialized loop with counter variables at some point. Happy hacking!

Here's a quickie for you. The other day I was working on a little coding kata, where the goal is to print out the verses to the drinking song "99 Bottles of Beer on the Wall". For the purpose of this demonstration, I'll narrow the problem down to just the first five verses.

Fresh off of making an episode about Ruby ranges, I automatically typed up some code that looked something like this.

(99..95).each do |n|
  puts "#{n} bottles of beer on the wall, #{n} bottles of beer"
  puts "Take one down, pass it around, #{n-1} bottles of beer on the wall"
end

I executed this code, confident I would get the result I expected. And then I was very surprised when nothing at all came out.

What I had forgotten was that Ruby ranges are one-way: they only go up, never down. When we expand this range to an array, it is empty:

(99..95).to_a                   # => []

To be perfectly frank, I consider this an omission in Ruby. Other languages have range features that can go either up or down. This is one of those rare cases in Ruby where typing out the code we imagine should work results in a failure.

It is very rare to see traditional C-style for loops in Ruby code, and it would be jarring if we had to resort to one now. Fortunately, Ruby doesn't leave us completely out in the cold in this situation. Instead of a Range, we can use the #downto method. We send it to the starting integer, and provide the ending number as an argument. Our block will now be called with the number 99, then 98, then 97, 96, and finally 95.

99.downto(95) do |n|
  puts "#{n} bottles of beer on the wall, #{n} bottles of beer"
  puts "Take one down, pass it around, #{n-1} bottles of beer on the wall"
end

# >> 99 bottles of beer on the wall, 99 bottles of beer
# >> Take one down, pass it around, 98 bottles of beer on the wall
# >> 98 bottles of beer on the wall, 98 bottles of beer
# >> Take one down, pass it around, 97 bottles of beer on the wall
# >> 97 bottles of beer on the wall, 97 bottles of beer
# >> Take one down, pass it around, 96 bottles of beer on the wall
# >> 96 bottles of beer on the wall, 96 bottles of beer
# >> Take one down, pass it around, 95 bottles of beer on the wall
# >> 95 bottles of beer on the wall, 95 bottles of beer
# >> Take one down, pass it around, 94 bottles of beer on the wall

Like so many other iterative methods in Ruby, #downto can also return an Enumerator. If we omit the block, we can then chain on whatever Enumerable method we like, such as #map.

99.downto(95).map{ |n| "#{n} bottles of beer" }
# => ["99 bottles of beer",
# "98 bottles of beer",
# "97 bottles of beer",
# "96 bottles of beer",
# "95 bottles of beer"]

It probably goes without saying that the #downto method is mirrored by an #upto method, which does exactly what it sounds like.

1.upto(5) do |n|
  puts n
end

# >> 1
# >> 2
# >> 3
# >> 4
# >> 5

Sadly, neither of these methods accepts a step unit, so they are only helpful in cases where we want to count up or down in increments of one.

If we want to get fancier, such as using custom increments or generating infinite sequences, there is a way to do that. But that's a topic for the next episode. Until then: Happy hacking!

Let's say we run a subscription screencasting service. Some of our episodes are members-only, and some of the episodes are free for anyone to watch. On our website's home page, we'd like to feature a random selection of the free videos so that visitors can get a sense of the content.

But we don't want the selection to be completely random. In particular, we don't want to accidentally show just videos from a particular miniseries. We want to ensure that the featured samples are picked from a broad range of the show's history.

Here's one way we might handle this. We define the number of sample episodes to show on the front page - let's say 2. Then we calculate a slice size from the total number of free episodes divided by the number of samples to feature. With this number in hand, we can use #each_slice to split the free episode list into equal-sized bins. Each bin represents a different period of time in the show's history.

We can map over those bins, and pick a single random episode out of each one with #sample. After flattening the result, we have our selection of episodes. It is random, but random in such a way that it is impossible for two brand-new or two very old episodes to be selected.

eps = [7, 11, 17, 20, 24, 27, 35, 42, 45, 46]

slice_count = 2
slice_size  = eps.size / slice_count # => 5

slices = eps.each_slice(slice_size).to_a
# => [[7, 11, 17, 20, 24], [27, 35, 42, 45, 46]]

picks = slices.map { |s|
  s.sample(1)                   # => [11], [27]
}.flatten
# => [11, 27]

Of course, it would still be possible for two episodes to be selected from right in the middle of the show's run. In order to ensure a better spread, and because we like the idea of having more featured videos on the homepage, we increase the slice count 3.

Unfortunately, this messes everything up. Because we currently have 10 free episodes to work with, the set doesn't divide evenly into three parts. As a result, when we tell the array to slice itself into 3-element arrays, it also generates a fourth one-element array with the remainder.

eps = [7, 11, 17, 20, 24, 27, 35, 42, 45, 46]

slice_count = 3
slice_size  = eps.size / slice_count # => 3

slices = eps.each_slice(slice_size).to_a
# => [[7, 11, 17], [20, 24, 27], [35, 42, 45], [46]]

picks = slices.map { |s|
  s.sample(1)                   # => [7], [20], [45], [46]
}.flatten
# => [7, 20, 45, 46]

We decide to introduce a special case for this eventuality. If the actual number of slices is greater than the intended slice count, we will do a bit of extra processing. First, we'll find the slice that should have been the final one. Next, we'll remove the leftover slice using pop, and assign that removed slice to a variable. The #pop method, if you're not familiar with it, removes the last item in an array and returns the element it popped off.

Then we'll concatenate the leftovers onto the slice before it.

The result is a set of 3 bins, as originally intended, where the fourth bin has one extra element. We can sample these bins as before, and get the expected number of final picks.

if slices.size > slice_count
  end_slice = slices[slice_count - 1] # => [35, 42, 45]
  leftover  = slices.pop              # => [46]
  end_slice.concat(leftover)          # => [35, 42, 45, 46]
end

slices
# => [[7, 11, 17], [20, 24, 27], [35, 42, 45, 46]]

picks = slices.map { |s|
  s.sample(1)                   # => [17], [20], [45]
}.flatten
# => [17, 20, 45]

This works well, but we hate to introduce a conditional into this algorithm. Conditionals make code harder to reason about, because now in addition to thinking about what each line of code is doing, we also need to think about whether a line of code is being executed at all.

Let's see if we can get rid of this conditional while still handling all cases. We'll do this progressively, moving more and more code out of the conditional until hopefully we can do away with it entirely.

First off, we don't technically need the line which finds the ending slice to be inside the if block. We move that line out.

end_slice = slices[slice_count - 1] # => [35, 42, 45]
if slices.size > slice_count
  leftover  = slices.pop              # => [46]
  end_slice.concat(leftover)          # => [35, 42, 45, 46]
end

For the next move, we need to understand a bit more about how #pop works. If we just send it with no arguments, it returns the last element. Or nil, if there are no elements.

But if we supply an integer argument, #pop snips off and returns an array of that many elements. 1 gets us an array of one, 3 gets an array of 3, and so on. If we supply an argument of zero, we get an empty array back—/not/ a nil value.

a = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
a.pop                           # => 10
a                               # => [1, 2, 3, 4, 5, 6, 7, 8, 9]

[].pop                          # => nil

a.pop(1)                        # => [9]
a.pop(3)                        # => [6, 7, 8]
a.pop(0)                        # => []

With this knowledge in mind, we move on to the next step. We initialize a leftover_count variable to zero. Inside the if statement we reset it to 1. Then we use the leftover count as an argument to pop, and flatten the resulting array.

leftover_count = 0
if slices.size > slice_count
  leftover_count = 1
  leftover  = slices.pop(leftover_count).flatten # => [46]
  end_slice.concat(leftover)          # => [35, 42, 45, 46]
end

At this point, we no longer need the pop and concat actions to be inside the conditional block. If there are no leftovers, the leftover count will be zero and the pop will return an empty array and leave the source array untouched. And of course concatenating an empty array to another array is a harmless no-op, so it's safe to execute the concat in either situation.

Accordingly, we move those two lines out of the conditional.

leftover_count = 0
if slices.size > slice_count
  leftover_count = 1
end
leftover  = slices.pop(leftover_count).flatten # => [46]
end_slice.concat(leftover)          # => [35, 42, 45, 46]

At this point, all the if statement is doing is deciding whether to set a variable to 1 or 0. We can simplify this using the ternary operator.

leftover_count = slices.size > slice_count ? 1 : 0
leftover  = slices.pop(leftover_count).flatten # => [46]
end_slice.concat(leftover)          # => [35, 42, 45, 46]

We've now reduced the conditional to a single line, but can we get rid of it completely? Well, remember what we need to do here. The result should be one if the slice size is one greater than the intended slice count, and it should be zero if the two are equal. When we phrase the problem this way, it rings a bell in the arithmetic centers of our brains: we can get the result we need by just subtracting the intended slice count from the actual size of the slices array.

eps = [7, 11, 17, 20, 24, 27, 35, 42, 45, 46]

slice_count = 3
slice_size  = eps.size / slice_count # => 3

slices = eps.each_slice(slice_size).to_a
# => [[7, 11, 17], [20, 24, 27], [35, 42, 45], [46]]

end_slice = slices[slice_count - 1] # => [35, 42, 45]
leftover_count = slices.size - slice_count     # => 1
leftover = slices.pop(leftover_count).flatten # => [46]
end_slice.concat(leftover)          # => [35, 42, 45, 46]

slices
# => [[7, 11, 17], [20, 24, 27], [35, 42, 45, 46]]

picks = slices.map { |s|
  s.sample(1)                   # => [17], [24], [46]
}.flatten
# => [17, 24, 46]

When there's a leftover slice, the leftover count will be 1 and the leftovers will be moved into the last full bin. If we switch the desired slice count to one that evenly divides the list, the leftover count is zero and the concatenation has no effect.

At this point, we have achieved our goal: we've found a way to divide a set of videos into equal or nearly equal sized bins, even when the division results in a leftover element. And we've done it without any conditionals at all. Along the way, we've seen that the #pop method has more to it than just removing and returning single values. Happy hacking!

Hey folks! I'm experimenting with putting the script in the episode body this time. Not sure how this will work out with RSS readers, etc. Let me know what you think!

We cover a lot of slightly obscure features of Ruby and its libraries on this show. Some of these capabilities may seem like they only have very niche use cases. Today, I want to show an example of how, by putting together a bunch of little bits of Ruby knowledge, we can solve a pragmatic application programming problem.

Let's say we are working on the latest and greatest social networking site. We're at the point where we want to start inviting beta users into the site, and so we need to send some emails out. We decide to separate emailing into its own gateway class. In this class we have a method called =#send_{invitationemail}. It takes a recipient address and some other data as parameters.

class EmailGateway
  def send_invitation_email(to:, **data)
    subject = "Invitation to join Squawker"
    body = "Hi, #{data[:name]}!\n" +
      "You're invited to join Squawker! To get started, go there: \n" +
      data[:signup_url]
    send_mail(to: to, subject: subject, body: body)
  end

  private

  def send_mail(to:, subject:, body:)
    puts "*** Sending mail to: #{to}"
    puts "Subject: #{subject}"
    puts
    puts body
  end

end

To invite a new member, we can instantiate a gateway object and send it the #send_invitation_email message, along with all the requisite extra data used to fill in the email template.

require "./email_gateway"

gw = EmailGateway.new

gw.send_invitation_email(to: "marvin@example.org",
                         name: "Marvin the Robot",
                         signup_url: "http://example.com/signup")

# >> *** Sending mail to: marvin@example.org
# >> Subject: Invitation to join Squawker
# >>
# >> Hi, Marvin the Robot!
# >> You're invited to join Squawker! To get started, go there:
# >> http://example.com/signup

With beta invitations working, we realize we now need to send out welcome messages as well. In adding another type of mailing, we realize we don't want to duplicate the ugly string-concatenation approach of our first method. We also feel like we should probably generalize our mail-sending system a bit.

But up till now this has been a very lightweight app. It's built on Sinatra rather than Rails, and we don't have an industrial-strength mailer framework all set up and ready to go.

As we think through the requirements of a generalized mail-sending subsystem, based on what we know about existing frameworks such as ActionMailer, we start to get discouraged. If we're going to be writing many more email bodies, we don't want to embed them as strings… we'll probably want to write them in their own template files… so we'll have to write some code to find the right template file for a given mailing… and then we'll need a way to personalize each template, so we'll need to hook in a templating system like ERB or mustache…

Hang on, hang on. Let's take a step back for a moment. We're still only talking about two different email templates. And we're not worried about sending both HTML and text versions of the emails or anything else fancy like that. So let's see if we can use what we know about Ruby and come up with a pragmatic solution. Something that makes it a little more pleasant to write new types of mailing, without overthinking things too much.

Here's what we come up with. First off, we bring in our unindent method from episode #249. Then we write a new method, #send_welcome_email. Like the other mailing method, it requires a to keyword, and can accept an arbitrary set of other data attributes.

We start it similarly to the last method, by defining a subject. Then we define a body. This is where we diverge from the other mailer method. Instead of a quoted string, we start a heredoc, filtered by our unindent method to get rid of unwanted leading spaces. Then we apply the format operator to the unindented heredoc. Let's complete the heredoc before we talk about the format operator and its other argument.

Inside the heredoc, we define the text of the message. Since the whole doc will be unindented, we are free to indent the body consistent with our usual coding style. Anywhere in the body where we need a personalized value, we enclose its name in curly braces preceded by a percent sign. The personalizable elements in this message are the recipient's name, and a URL for the site homepage.

Once we have the subject and body defined, we use the private send_mail method just as in the other mailer method.

Now that we are done defining the method body, let's talk a bit more about how the personalization will work. As you may recall from episode #194, Ruby strings have a format operator, the percent sign. This operator performs printf-style expansions in the string, using the supplied arguments. When we pass a hash instead of an array to the format operator, Ruby treats the hash as a source of values for named expansion keys. Everywhere the text contains a curly-braced word preceded by a percent sign, the format process looks up that word in the given hash of data and replaces the whole specifier with the value it finds there.

The upshot is that string formats give us a quick and dirty way to implement templated text without resorting to templating engines.

class EmailGateway
  def unindent(s)
    s.gsub(/^#{s.scan(/^[ \t]+(?=\S)/).min}/, "")
  end

  def send_invitation_email(to:, **data)
    subject = "Invitation to join Squawker"
    body = "Hi, #{data[:name]}!\n" +
      "You're invited to join Squawker! To get started, go there: \n" +
      data[:signup_url]
    send_mail(to: to, subject: subject, body: body)
  end

  def send_welcome_email(to:, **data)
    subject = "Welcome to Squawker!"
    body    = unindent(<<-'EOF') % data
 Hi %{name}! Welcome to Squawker!

 Your account is all set up and ready to go! Access it anytime
 at: %{home_url}
 EOF

    send_mail(to: to, subject: subject, body: body)
  end

  private

  def send_mail(to:, subject:, body:)
    puts "*** Sending mail to: #{to}"
    puts "Subject: #{subject}"
    puts
    puts body
  end

end

Let's give our new mailer method a try. We supply a recipient email address, a name, and a home URL. When we look at the output, we can see that everything has been filled in where it should be.

gw = EmailGateway.new
gw.send_welcome_email(to: "crow@example.org",
                      name: "Crow T. Robot",
                      home_url: "http://example.com")

# >> *** Sending mail to: crow@example.org
# >> Subject: Welcome to Squawker!
# >>
# >> Hi Crow T. Robot! Welcome to Squawker!
# >>
# >> Your account is all set up and ready to go! Access it anytime
# >> at: http://example.com

As it turns out, we haven't really generalized our mail gateway at all. What we've done is come up with a new convention for writing mailer methods that lets us avoid ugly and typo-prone string quoting and concatenation. It also allows us to insert personalized values into the templates by name, in a way that is decoupled from where the values came from.

Once we write a few more mailer methods, we might feel the urge to take another pass and factor out some of the commonalities between our mailer methods. If we write even more mailing types, we may eventually look for a way to push the mail templates out into their own files rather than expanding this file indefinitely. But for right now, we've found a way forward that starts us on an incremental path to greater generality, without becoming overwhelmed by all the requirements of an elaborate mailing subsystem. Happy hacking!

Today we tackle one of Ruby's most controversial new features. Building on the example from episode #249, we'll look at how to extend a core class with custom methods, without attracting the kinds of potential bugs that "monkey-patching" invites.

In today's episode we explore methods for unindenting text. Along the way, we'll learn some tricks and play some code golf.

In this episode, we'll look at an easy way to find items in a collection with the lowest (or highest) of some attribute. And at how we can apply this knowledge to more than just arrays.

Today we'll compare a few different ways to quote large, multi-line blocks of text.

Today's dish is a quick little refactoring that demonstrates working with multiple values pulled from a Hash.

An easier way to find the location of the current source file.

In today's episode, we'll compare two methods for getting to the body of an email: one imperative; one functional; and both idiomatic Ruby.

In this episode we take a method with a large and unweildy signature, and begin to pull parameters out into configurable options.

Today we discuss the downsides of using require_relative, and explore a more robust way to handle code loading within projects and libraries.

In this episode, we look into how to get the most out of the Regexp#match method.

Today our ongoing series on loading code brings us to the topic of loading features relative to the current file.

Think you know everything there is to know about parameter defaults? Think again!

In this episode we dig into RubyGems in order to understand how it augments Ruby's built-in code loading mechanisms.

As we continue to talk about the process of loading code in Ruby, today we look at how requiring features differs from loading files.

As we continue our exploration of how code is loaded into Ruby processes, today we take a look at a technique for running an external script inline without polluting the current Ruby context.

Today's episode kicks off a little series on loading code in Ruby. To start off, we'll look at what it would take to recreate Ruby's logic for locating code files to be loaded.

Today we talk about how to let client coders know that something might be amiss... the Ruby way.

Continuing from the last episode's them of exploring the Range type, today we look at one of Ruby's more peculiar legacies from AWK and Perl.

Good morning diners! On today's menu, we have a deep exploration of Ruby's Range type. Even if you've used ranges before, you might find something new to chew on. Enjoy!

Today, some options for getting the first N elements from a list.

Last week we talked about the consequences of building an app without unit tests that might have forced us to deal with internal coupling. Today we look at the same app from a new perspective.

In today's episode we'll meet an application written without tests, and discover the repercussions of that decision.

Regular expressions are one of the most powerful tools at our disposal. But sometimes they aren't as well suited to a job as they may first appear. In today's episode we look at an alternative to regexen for validating strings.

Today we'll look at two different ways to memoize a complex method, and talk about why I prefer one over the other.

In which I rant a bit about the dangers of "monkey-patching".

As our series on representing physical quantities draws to a close, we turn our attention to existing libraries, specifically the Unitwise gem.

It's often desirable compare objects to see which is "lesser" or "greater"; but not all objects are inherently comparable. Today we'll look at a scheme for easily imposing an arbitrary ordering on a constrained set of value objects. 

We know that Value Objects are useful; today we'll meet a gem that makes them easier to build.

Today we look at a lesser-known but handy pair of methods on Strings.

Since Ruby 2.1, def returns a symbol. In this episode we'll take a look at why this matters, and how it might change the way we define methods.

Where do we draw the line between objects differntiated only by state, and objects differentiated by their class? That's the question we'll examine in this episode.

Today we learn about Ruby gem that makes it easier to build immutable value objects.

In episode 205 we introduced the spaceship (<=>) operator, but we also introduced an incompatibility with how Ruby's builtin comparisons work. Today we'll address this oversight.

Today, a story about a refactoring that went south, and turned out not to be a refactoring at all.

In today's episode, we explore a practical application of the famous "tell, don't ask" principle.

Today we look at Ruby's grep method, and explore the idea of matching objects by example.

Today we puzzle through a design problem that threatens to result in dozens of extra methods. 

Today's topic is "conversion protocols", an extensible way to enable safe, automatic conversions between types.

Today's episode deals with a small matter of style, one that can have an impact on how easy it is to refactor code.

The distinction between "public" and "private" method visibility in Ruby is pretty obvious. But when should we use "protected" visibility? This episode attempts to answer that question.

Have you ever wondered why Ruby has both #to_i and #to_int methods? Or both #to_a and #to_ary? In today's episode we'll answer this question, and look at how we can use implicit conversion methods to our advantage.

In this episode we look at how to convert our Feet objects back to core numeric types. In the process, we gain the ability to use our Feet objects in format strings.

For core types like Float, Ruby has both conversion methods (#to_f), and conversion functions (Float()). When should we use one vs. the other? This episode attempts to answer that question.

In this episode we create a function for converting arbitrary values into Feet objects.

Ruby generally doesn't allow mixing of types without explicit conversion. So have you ever wondered why it's possible to multiply 2.3 (a float) by 5 (an integer)? In today's episode we'll discover how Ruby's implicit coercions work, and how to apply them to our own custom numeric-like classes.

In this episode we meet a Ruby standard module that makes it easy to make classes comparable and sortable.

Building on the previous episode on hash tables, today we tackle the concept of hash equality: how Ruby decides if two objects are the same for the purpose of use as hash keys.

Today's dish is an exploration of how Ruby is able to quickly look up keys in hashes.

Today's episode deals with the concepts of identity and equality - what makes one object equal to another.

In this episode we learn how mutability can lead to bugs in Value Objects... and how we can eliminate these types of bugs once and for all.

In today's episode we take a look at a storied problem in software: defects caused by accidentally mixing-up the units of measurement used in calculations. And we kick off a multi-part series exploring how to build objects that represent physical quantities.

Today's episode demonstrates an easy way to build big regular expressions out of little ones.

In today's episode we look at a potential complication of using the Decorator pattern, and discuss how to resolve it by enforcing Command/Query Separation.

Today we explore the Decorator pattern, with the help of the SimpleDelegator standard library.

Have you ever wanted a way to customize the format of certain generated strings, but felt like ERB or some other templating language was overkill? If so, today's episode should satisfy!

Following on from the last episode, today we look at some more advanced uses of Ruby's string formatting features.

Sometimes Kernel#puts and string interpolation doesn't give us the level of control we want over our program's output. When we need to control numeric formatting and field widths, we need to understand string formats. And that's what this episode is all about!

While the File methods are great for occasional use, for programs that deal extensively in filenames we need to bring out the big guns. Today we'll get an overview of Pathname, Ruby's swiss army knife for path manipulation.

Ruby provides a lot of tools for breaking filenames into their component parts, but they aren't always well documented. Today's episode combines goes over some basics as well as a tip you might not be familiar with.

When loading domain objects from an external service, it can be expensive to load up associated objects with them. Today's episode takes a look at a pattern for transparently lazy-loading such associations.

Today we look at the humble String#gsub method, and learn that it has some surprising tricks up its sleeve.

In today's episode, we'll examine how the availability of tools to aid refactoring can change how we write code.

Today's episode is about a semantic convention for error handling that I learned from Jim Weirich.

Today we go a little deeper into keyword arguments in Ruby 2.0/2.1, covering a few use cases we didn't cover in the last episode.

Today's episode is a guide to transitioning various hash argument idioms to Ruby 2.0/2.1 keywords.

Today's dish is a refactoring approached from two different perspectives. Enjoy!

Recently we looked at the Sequel library for interacting with SQL databases. Today, we'll use Sequel again to play with the native JSON support in PostgreSQL 9.3.

We've used the "ghost object" pattern to lazily load attributes of a model object. And we've made a macro to easily declare "ghost-loadable" attribute accessors. Today we complete the generalization of ghost loading by extracting a module that makes it easy for any model object to declare lazily-loaded attributes.

When is it appropriate to metaprogram? Today's episode looks at one situation in which it may be a good choice.

Today's episode demonstrates a technique for speeding up the process of sorting some collections.

In today's episode we learn at how to implement lazy loading using the "ghost object" pattern.

ActiveRecord has become practically synonymous with SQL database access in Ruby, but it's not the only way to talk to SQL stores. Today we'll explore Sequel, a wonderfully rich tool for interacting with many different SQL RDBMSes.

In the last episode we looked at the problem of "aliasing", here there are multiple objects representing a single row in a database. Today, we'll look at one possible solution to that problem.

Notes:

• DataMapper: http://datamapper.org/
• Ruby Object Mapper (ROM): http://rom-rb.org/
• Perpetuity: https://github.com/jgaskins/perpetuity

Today we look at a perncious problem that sometimes plagues code which uses an Object-Relational Mapper (ORM).

Documentation of the ActiveRecord inverse_of option mentioned in the episode can be found here: http://api.rubyonrails.org/classes/ActiveRecord/Associations/ClassMethods.html

In this episode we look at a tool that I've found invaluable for managing configuration in my applications.

In today's episode I cover a classic development technique that doesn't get enough press these days. Interactive, exploratory development was and is common in the Lisp world, and Ruby's dynamic nature is aptly suited to take advantage of this style of programming. Today I'll show you how I used REPL-driven development in Pry and Emacs to begin to build a video export tool.

By the way, right after I made this episode I discovered that Conrad Irwin had done a RubyConf talk on REPL-driven development. You can watch it here: https://www.youtube.com/watch?v=D9j_Mf91M0I

Today's special is a note on style with a side order of opinion. Enjoy!

(Note: I made a mistake in the visualization of which variables correspond to which on either side of the equals sign. Which kind of illustrates my point!)

Newer arrivals to Ruby sometimes wonder when to use #each, and when to use 'for'. Today, a detailed explanation of how they differ, and an opinion on when to use 'for'.

Today's dish is another one from one of my favorite cookbooks: Martin Fowler's Patterns of Enterprise Application Development. This time, we'll use the Registry pattern to make it easy for objects to find the collaborators they need, without hardcoding their dependencies.

Even the most basic of Ruby methods sometimes have hidden depth. Today we'll take a look at some advanced uses of the puts method.

Today's episode covers an incredibly handy capability of Ruby's hashes that I didn't know about until recently.

You might remember the Gateway and Mapper patterns from recent episodes. Today we'll look at how to insert a caching layer between those two patterns. In the process, we'll examine how pattern-based design decisions can make it easy to add new functionality without changing existing classes.

In this very special episode, guest chef Pat Shaughnessy takes us on a whirlwind tour of the Ruby internals that make the Enumerable#all? method tick.

When debugging a tricky problem, don't you sometimes wish you could just drop a line of debugging code right in the middle of a third-party gem? This episode shows how to do this in a responsible fashion.

Ruby doesn't have a built-in concept of "abstract" classes, but sometimes we want a way to show other programmers when methods are left as an exercise for the implementor. This episode discusses how.

In previous episodes, we got the Tapas::Queue class under test, using a couple of different thread-testing techniques. Now that it has tests, it's time to refactor.

The steps of this refactoring can be seen on this Github branch: https://github.com/avdi/tapas-queue/commits/refactor-conditions

Today we explore a pattern for bridging the gap between different domain models.

Today we follow up on the last episode to talk about YAML::Store. It's like PStore, only with YAML!

In today's episode we take a look at PStore, a simple but capable persistence mechanism that ships with Ruby.

Today's episode introduces the concept of thread-local variables, and shows how they can be put to use in an ActiveRecord-like library.

Today we revisit the Enumerable#reduce method, addressing some viewer feedback about seed values as well as exploring a novel application of reduce for traversing data structures.

Sometimes we'd like to treat Ruby arrays like sets, in which each item is unique. Today's dish shows how!

This episode takes a look at some potentially surprising rules for how Ruby looks up constants.

Notes:

• "Everything you ever wanted to know about constant lookup in Ruby", by Conrad Irwin

In this episode we'll explore a novel technique for testing multithreaded code.

The lockstep library can be found here: https://github.com/avdi/lockstep

Today, special guest chef James Edward Gray II hosts, and shows us a thing or two about generating arrays pre-filled with values!

Today's episode delves into some advanced regular expression features, including "lookahead" and "lookbehind".

Notes for further exploration:

• The book "Programming Ruby 1.9 and 2.0"
• rubular.com
• Talks from Nell Shamrell. There are also slides.
• Nell's blog series on regular expressions in Ruby

(Clip from "Beneath the Surface: Regular Expressions in Ruby", CC by Nell Shamrell. Recorded by Confreaks at Lone Star Ruby Conference 2013)

Working with threads is hard enough, but getting them under test is even trickier. In this episode we start to look at techniques for verifying the logic of multithreaded code.

We try to avoid writing slow unit tests, but what if we are testing a method whose responsibilities include waiting for a period of time? In this episode we look at a few approaches, talk about the difference between testing logic and testing system interactions, and finally settle on a strategy that leads us to a more flexible design.

Today we look at a gem that can make command-line scripts more pleasant to use.

Today's episode tackles a subject I've always wondered about: how accurate is Kernel#sleep?

Sooner or later you'll need to generate statistics from a collection. In this episode, we look at how to produce min, max, sum, average, median, and standard deviation from a set of samples.

Today's episode takes on a simple task--summing up a list of numbers--and uses it to demonstrate both the Enumerable#reduce method as well as Symbol#to_proc.

This episode looks at how to invoke Rake tasks from other programs.

Today's is another threading episode. This time around, we tackle the subject of "atomicity", and learn about the false assumption at the root of many threading bugs.

Today we learn about the concept of a "recursive mutex" as we help a turtle-racing league update their software systems.

Continuing with the general theme of threads and concurrency, today we look at another pattern for splitting up work in parallel.

In this episode we take some already-good code and make it even better.

In this episode we finally discover why the Timeout module is unsafe, and a facility new in Ruby 2.0 that makes dealing with thread interruptions much less error-prone.

In today's episode, we'll simplify our thread-safe queue code by employing a "benign value" to represent the default max queue size.

We continue to evolve our thread-safe queue implementation. Today, we add the ability to set a bound on the queue size.

In this episode some bugs turn up in our thread-safe queue class.

Continuing to build our own thread-safe queue class, today we give it the ability to handle time out enqueues and dequeues.

As we continue to explore the fundamental building blocks of multithreaded programming, today we encounter the Condition Variable, and what it has to do with my local delicatessen. 

In this episode we introduce the concept of a "critical section", and learn about one of the fundamental primitives that makes multithreaded programming possible.

Concurrent programming is hard, and one of the things that makes it so hard is that a thread can unexpectedly die without giving the programmer any indication. In this episode we look at some ways to make threads fail fast and loudly while in development.

In this, the final (for now) episode of the Rake miniseries, we take a look at how to speed up Rake runs by taking advantage of multiple cores.

My children know they are supposed to clean their place at the table after dinner. Sometimes software builds need to be cleaned up too, and today we'll learn about an optional Rake library which streamlines this process.

Rake has a lot of tricks up its sleeve. In this episode we'll look at some of the helpers it provides for performing various common operations on files.

Today's dish is a real delicacy... we're going to dig into one of Rake's most powerful but little-known capabilities and see how we can easily transform collections of path names with the #pathmap method.

In this episode we learn how to write advanced Rake rules which programatically determine the appropriate prerequisites for a given target file.

As we continue our series on Rake, today we look at the Rake::FileList and how it can help us find the files we need and ignore the ones we don't.

Today we begin a series on the Rake build tool. We'll be getting into some powerful, lesser-known features as we continue on; but we'll start out with a quick review of Rake basics.

Queue, somewhat surprisingly, is not enumerable. Which is all the excuse we need to have some more fun with the Enumerator class, in today's episode.

Last time around we introduced the Queue class. In this episode, we'll put it to work!

Today's episode introduces the Queue standard library, and shows how it can be used to coordinate threads.

Ruby has both symbolic and English forms of the logical "and" and "or" operators. Which one to choose may seem like a matter of taste, but that assumption can get you into trouble.

Today we take a brief sojourn out of Ruby and into the Elixir programming language. I'll show you one of my first programs in Elixir and point out how the language's pattern-matching abilities can contribute to some very elegant solutions.

If this episode piques your interest in Elixir, here are some resources you can explore to find out more:

• The Elixir home page: http://elixir-lang.org/
• Meet Elixir, an excellent screencast from Peepcode: https://peepcode.com/products/elixir
• Programming Elixir, a book in beta by Dave Thomas: http://pragprog.com/book/elixir/programming-elixir

Have you ever had a project that spat out so much deubg output when it ran that you couldn't tell when it had a legitimate error or warning to report? In today's episode I'll show you how to quickly and easily track down the source of unwanted output.

Last episode we looked at a simple way to test if a method executes a block as intended that will work in any test framework. But if your tool of choice is RSpec, there are some matchers that make these kinds of tests shorter and more declarative.

Today's episode answers the question: how do you test that a block passed to a method is executed as intended?

In today's episode I tackle the question of how many tests are enough. I'll show how for me, it's all about how long it's been since I last got new information from a test.

In this episode, we examine the pros and cons of a few of techniques for making boolean flags passed to methods read more meaningully.

A small dish today, about some numeric methods I often forget exist.

In another peek into my ongoing behind-the-scenes work on RubyTapas, today I extract the responsibility for representing a logged-in screen-scraping user-agent session into its own class.

In today's live-recorded episode I show one of the most fundamental refactorings for breaking up a large class into smaller classes.

Since this is a live episode there is no transcript today. The Naught codebase can be found at http://github.com/avdi/naught

In this quick live episode I demonstrate a standard library that improves on Kernel#p

In today's episode, we encounter a starship in trouble, and a software pattern that helps cleanly disable whole categories of behavior.

We all use 'puts' for debugging at some point. This episode shows why 'p' is a better choice.

In this episode we take a look at the Special Case pattern, and see how it can be used to drastically simplify some typical logic around logged-in and anonymous users.

Nobody wants to debug an "undefined method for NilClass" error. Today's episode shows a little trick for making these errors more meaningful.

A refactoring episode, showing how to replace exceptions-as-flow-control with the "catch" and "throw" methods.

This episode takes a look at how to use the Nokogiri gem to extract data from large HTML documents without reading the whole document into memory.

This episode explores some of the many ways we can come across a nil value, and why that's a problem.

In today's episode: super-powered string-munging with regexes and the square-bracket operator.

Today, some thoughts on writing class-level attribute accessors.

In this episode we'll mimic the behavior of command-line tools like Git that automatically page their output if they detect that they are being run from a terminal.

It would be great if time were always represented in readable, un-ambiguous ISO8601 formats. But  when importing legacy data we don't often have that luxury. In this episode, we look at some tools for parsing various time and date representations.

Finishing up my list of TODO items, today I address some structural coupling that my test stubs have revealed.

This is a live episode, so no script today.

Note: This week RubyTapas moves to its new release schedule of two episodes a week. I've already sent out an update detailing the reason for the change, but for those who have disabled email updates, I've also added a note to the FAQ here: http://www.rubytapas.com/faq#frequency-change

Working through my TODO list from the recent feature additions, in today's episode I seek to resolve the semantic conflict between User and GemUser through refactoring.

This is a live episode, so no script today.

Code here: https://github.com/avdi/gem-love/tree/rubytapas-episode-102

Not all refactoring is about eliminating duplication. In today's episode we'll refactor some code for the purpose of revealing intent to future readers.

Today's episode tackles the problem of talking to a web app which has no published API, using the Mechanize screen-scraping gem. In the process, we'll explore the Gateway pattern for encapsulating external resources.

Today the focus is on a convenient way to extract substrings using regular expressions. We glossed over this technique briefly in an earlier episode, but now it's front and center.

In today's concluding slice of this live-coding session, I wrap up my work on adding users to Gem-Love by TDDing the server side support for API keys.

This is a live episode, so no script today.

Source: https://github.com/avdi/gem-love/tree/rubytapas-095

As I continue to add the concepts of users to Gem-Love, I tackle the problem of making the client side auth-token aware.

This is a live episode, so there is no script today.

Source: https://github.com/avdi/gem-love/tree/rubytapas-095

In this continuation of episode 95, I finish writing new acceptance test helpers, and get to my first proper test failure.

This is a live episode, so no script today.

Source here: https://github.com/avdi/gem-love/tree/rubytapas-095

Diving back into my Gem-Love project, today I begin the process of adding users to the system. Because this is a larger feature, it will be split across multiple episodes. However, since they are all parts of the same programming session, I'll be running them back-to-back to preserve continuity.

This episode was cut from a live recording, so there is no script available.

The code can be found here: https://github.com/avdi/gem-love/tree/rubytapas-095

Have you ever found yourself looking for a '#to_bool' method in Ruby? This episode explores why boolean conversion isn't needed as often as you might think; as well as how to accomplish it when it IS called for.

A common question on Ruby forums and mailing lists is: "why is there no Boolean data type?". This episode attempts to answer that question.

Katrina Owen contributed an example of coincidental duplication I liked so much I decided to make a second episode about it. Enjoy!

In this, the first (but far from the last!) RubyTapas episode to focus on a Ruby 2.0 feature, we look at the implications of Ruby's newly relaxed rules for binding method objects to objects.

Ruby gives as a number of options when it comes to defining class methods. This episode starts by demonstrating three of them, and then gets a bit opinionated.

The DRY principle teaches us to get rid of duplication wherever we find it. But not all duplication is created equal...

In this continuation of the Gem-Love project, I complete the end-to-end acceptance test, verifying that the client can talk to the server.

Source code here: https://github.com/avdi/gem-love/tree/rubytapas-088

Just a reminder, I'm collecting feedback so I can make RubyTapas even better! If you want your voice to be heard, fill out the survey: http://shiprise.wufoo.com/forms/r7x2q5/

Naming things is hard, and it's difficult to come up with hard-and-fast rules for it. But it can be helpful to learn from examples of real-world naming decisions that led to beneficial design changes. In this episode, we'll look at a method name change that clarified object responsibilities in a program dealing with live events.

In this episode we tackle how to override base class methods while cleanly passing through multiple base-class arguments.

Notes:

• Paul Battley on Destructuring Assignment in Ruby

Sometimes a block or method may receive arguments that you just don't care about. This episode introduces the idiomatic Ruby way to indicate that certain arguments should be ignored.

Today we take a look at a related feature to Ruby's "splat" operator: the ability to recursively destructure arrays using grouped assignment.

In today's episode we discover how to make an arbitrary object implicitly "splat"-able.

It's a small thing, but judicious use of inline assignment can make code more DRY and concise. In this episode we look at how to do it, and when.

Delving further into Ruby's destructuring assignment, or "splat", mechanism, today we look at some cases where Ruby performs splatting without an explicit '*' operator.

Ruby supports a limited form of destructuring assignment, in the form of the "splat" operator. In today's episode we go over splat basics, in order to lay a foundation for more advanced splatting techniques to come.

There is more than one way to concatenate arrays, but not all ways are created equal. In this episode we'll compare three approaches in terms of both semantics and efficiency.

We all know Java is the best programming lanagueg out there, but maybe you've seen the literal syntax some lesser languages have for initializing data structures like arrays and maps. In this episode we'll look at a technique for concisely initializing Java data structures with values.

In a final refactoring to our pseudo-tail(1), we use an enumerator to encapsulate the process of searching text chunks for newlines.

Continuing to refactor our minimal tail(1) implementation, today we clean up a loop by encapsulating its state in a new object.

So far, our reimplementation of tail(1) bears a striking resemblance to the style of code we might find in the C implementation of the same utility. In this episode we'll make the code a little more Rubyish by extracting loops into block methods.

We've figured out how to read chunks of text backwards from a file, now it's time to tackle searching that text for the beginnings of lines.

Marching right along in our reimplementation of UNIX tail(1), today we tackle the problem of dumping the tail of the file to STDOUT once we've found the starting point.

As we continue to rewrite a subset of the UNIX tail(1) command, we learn how to write a do...while loop in Ruby.

This episode kicks off a short miniseries on re-implementing a small subset of the UNIX "tail" command in Ruby. We'll start by learning how to jump around to arbitrary positions in a file.

In the second of two episodes about the 'break' keyword, we discover how to override method return values to suit our own needs.

In this, the first of two episodes on the 'break' keyword, we look at how it can be applied beyond breaking out of loops.

The last Gem-Love episode had me creating a command-line client. This time around, I use Rack and Sinatra to build a simple server.

Source code can be found here: https://github.com/avdi/gem-love/tree/rubytapas-069

In some applications there is a many-to-many relationship between types of objects that may be displayed to users, and formats in which they may be rendered. There may even be multiple display styles within a given format: for instance, a "summary" style and a "detailed" style. In this episode well look at one possible option for decoupling what is being displayed from how it is displayed.

In the last episode we gave a class the ability to cache HTTP responses in a Hash or something that behaves like a Hash. Now we'll introduce the Moneta gem, a library that abstracts many different key-value stores behind a uniform, Hash-like interface.

Web services are slow to request and often have rate caps. A cache can cut down on requests to remote APIs. But how best to design the interface between the code that uses an API, and the caching layer? In this episode we'll explore that question.

This is the culmination of the last few PulseFFI episodes. I'll add some insurance to the high-level PulseFFI.mainloop method, and then make use of it in the proof-of-concept script. Finally, I'll discuss some of my philosophy for building library APIs.

Browse the source code here: https://github.com/avdi/pulse-ffi/tree/rubytapas-065

Sorry, no transcript for this live-recorded episode.

A number of Ruby standard library methods return an Enumerator if no block is passed to them. In this episode, we'll learn how to emulate this behavior in our own methods.

Returning to my Gem-Love project in this episode, I use TDD to drive out a client-side implementation for the first feature. In the process, I talk about message-first design.

Check out the source here: https://github.com/avdi/gem-love/tree/rubytapas-063

In this episode we'll explore Ruby's lightweight concurrency primitive by using it to emulate the Enumerator class.

Continuing in my extraction of a high-level API for the PulseFFI library, in this episode I test-drive the Mainloop.run, a composed method that brings together the work I've done so far.

Browse the source code here: https://github.com/avdi/pulse-ffi/tree/rubytapas-061

Sorry, no transcript for this live-recorded episode.

Now that we've been formally introduced to the Enumerator class, it's time to apply that knowledge in a concrete way. In this episode we use Pathname and Enumerator to locate a project-wide econfiguration file.

In previous episodes we've touched on Enumerator in passing. This time it'll take center stage, as we explore just what an Enumerator is and what it's good for.

This episode explores ARGF, a powerful object for working with input files in command-line programs.

In this episode I continue to felsh out the high-level API for my PulseFFI library, using TDD and mock objects to define how the object model interacts with the C bindings layer. I also compare the MiniTest and RSpec mocking/stubbing libraries.

Browse the source for this episode at: https://github.com/avdi/pulse-ffi/tree/rubytapas-057

Sorry, no transcript for this live-recorded episode.

In this, the most requested RubyTapas episode ever, we'll take a look at xmpfilter, the program I use to evaluate Ruby expressions from within my editor.

It's a library! No, it's an executable! Relax, your Ruby files can be both require-able libraries and executables!

In this edition of a miniseries on building a wrapper library using FFI, I begin to sketch out a higher-level API for starting up a PulseAudio mainloop.

Source code here: https://github.com/avdi/pulse-ffi/tree/rubytapas-054

Selectively running tests can be a lifesaver in a large codebase. This episode covers how to run just the tests you care about in both MiniTest and RSpec - including how to slectively run tests under Rake.

At the borders of our systems, mockist testing hits a point of diminishing returns. In this episode we take a look at when to stop mocking and start integration testing.

In this "live-style" episode, I begin to transition the PulseFFI proof-of-concept script into a Ruby library, using my smoke test to ensure everything continues to work.

Browse the source code here: https://github.com/avdi/pulse-ffi/tree/rubytapas-051

Sorry, no transcript for this live recording.

Sometimes a utility module is overkill. In this episode, we make a namespace module do double-duty.

Some functions are useful in many different contexts. In this episode we'll explore some ways to make them available both to library code and to client code of a library.

This episode covers how to write a "macro"-a method that generates or modifies other methods-using the classic example of memoizing method results.

In part 2 of this series, we buy ourselves some peace of mind with a simple smoke test.

The code for this episode can be found here: https://github.com/avdi/pulse-ffi/tree/RubyTapas047

In Part 2 of this series chronicling the creation of an app, I establish a test-driven rhythm, starting with a high-level acceptance test.

The source code for this episode is available here: https://github.com/avdi/gem-love/tree/rubytapas-episode-046

This cautionary episode demonstrates and explains a Hash gotcha that often comes as a surprise.

Somtetimes you need to know if exactly one element in a collection has a given property. For that situation, we have the #one? predicate method.

This episode looks at Ruby's logical XOR operator, and how it can be useful in checking that a method was called with the correct optional arguments.

Does code optimized for RAM usage need to be ugly? Find out, in today's episode!

Today's episode looks at a convenient tool for weeding through text and pulling out parts that match a given pattern.

In today's episode we'll try to come to grips with a slippery quality---a quality which is of importance to differentiating code that is easy to evolve, from code that isn't. Along the way, we'll write some Ruby code that looks disturbingly like Perl!

In this, the first of another ongoing occasional series, I revisit an open-source application I started three years ago and begin to rewrite it. Part 1 focuses on building a proof-of-concept RubyGems plugin.

The source code for this episode can be found on GitHub: https://github.com/avdi/gem-love/tree/rubytapas-episode-039

Other notes:

• How to write a RubyGem command plugin
• RubyGems API documentation

Sometimes it's hard to decide how to handle a failure in a method call. Raise an exception? Return a nil? Log an error? The best choice may differ depending on where the method is being called. In this episode, we look at a technique for defering the decision about how to deal with a failure to the point when a method is called.

Today's dish combines the great flavor of Ruby's 'case' statement with a little Proc seasoning and a threequals demiglace!

How is a Proc different from a Lambda? When do you use one over the other? And how to blocks fit into all this? Today's episode attemptes to answer these burning questions.

Many different kinds of Ruby objects respond to #call. Today's dish is a demonstration of how this fact can be harnessed to easily swap out different implementations of a collaborator object.

For such similar and broadly useful data structures, Struct and Hash are surprisingly incompatible. In this episode we look at some ways to smooth over their differences, and answer a viewer question in the process.

On the menu today is an exploration of what, exactly, happens when we define a class in Ruby.

Following on to the episodes about #fetch, today's episode answers the question: what do we do when we want the same defaulting behavior everwhere a Hash is used?

Today we revisit episode 21, and try out several different takes on registering observer callbacks inline.

Celebrate Friday with a stupid Ruby trick! In this epsiode we discover one of Ruby's most overlooked operators, and overload it just because we can.

This episode takes a look at two different methods for redirecting standard out and standard error - one simple and quick, one more comprehensive.

In episode 27, we switched from adding generated methods directly to a class, to using anonymous modules. In this episode, we consolidate our metaprogramming modules, and make them more self-documenting.

In this episode, we look at how to use dynamically generated modules to make metaprogrammed methods open for future extension.

With the FFI Gem, building wrappers for C libraries is easier than ever. This episode gives a brief overview of building the beginnings of a Ruby interface to the Linux libpulse library.

This episode focuses on OpenStruct, which makes it easy to create ad-hoc value objects from hashes.

Have you ever looked at a diff that was cluttered by meaningless "noise" - changes that existed just to satisfy the parser, and were a distraction from the semantic changes to the code? In this episode we look at some idioms and habits to help avoid these kinds of incidental changes.

In this episode we look at the Tempfile standard library, and how it can facilitate communicating with subprocesses.

Putting a rescue at the end of a line can get you into unexpected trouble. In this episode we'll look at why, as well as at the one case where an inline rescue makes sense.

In this episode we take a look at a common problem in Rails development: slimming down a controller whose functionality can't easily be pushed down into models. We address the problem using the Observer pattern and a tell-don't-ask design.

In previous episodes we've used the Struct class in passing to quickly construct classes. This time, we put Struct in the spotlight, and put it through its paces. You'll want to watch this one through even if you're familiar with Struct; you may find it has more tricks up its sleeve than you realized.

In today's episode, we revisit episode 11 (Method and Message), and take a look at a way to make the coupling between objects even looser, by making the name of the message one object sends to another variable.

Sooner or later you'll want to subclass Array. This episode looks at why that's a bad idea, and presents an alternative.

In this episode,  we explore command/query separation by looking at what happens to unit tests when the separation breaks down.

How to find out if there is a super method to be called, and other tricks for using super within a module.

In this final installment on the #fetch method we look at #fetch beyond Hash; #fetch with nested hashes; re-using default blocks; and more.

A dive into some of the dark corners of the `super` keyword.

Sometimes one instance is all you need.

How to use Hash#fetch to provide defaults for missing keys, and why you might prefer that to other techniques for defaulting values.

A look at the difference between methods and messages, and why passing methods around isn't as common in Ruby as it is in e.g. JavaScript.

How to reliably discover a user's home directory.

Some alternative ways to write symbols in Ruby code.

Hash#fetch is one of my favorite methods. In this episode, I take a look at using it to assert the existence of hash keys.

A look into how Ruby's object construction works, and how we can customize constructors for special scenarios.

A short introduction to using Ruby's Forwardable library for object composition.

Composing command lines with fancy array literals.

In this longer-than-usual episode, some thoughts on how to enable method logic to remain stable and unchanged while evolving and changing the source and scope of the values used by the logic.

Character literal syntax in Ruby.

How to format big numbers so that they are readable.

In this inaugural episode, a look at a handy syntax for writing out binary numbers.