---
layout: page
title: Overview
parent: Getting Started
nav_order: 2
---

# Polyphony - an Overview
{: .no_toc }

## Table of contents
{: .no_toc .text-delta }

- TOC
{:toc}

---

## Introduction

Polyphony is a new Ruby library for building concurrent applications in Ruby.
Polyphony provides a comprehensive, structured concurrency model based on Ruby
fibers and using libev as a high-performance event reactor.

Polyphony is designed to maximize developer happiness. It provides a natural and
fluent API for writing concurrent Ruby apps while using the stock Ruby APIs such
as `IO`, `Process`, `Socket`, `OpenSSL` and `Net::HTTP` in a concurrent
multi-fiber environment. In addition, Polyphony offers a solid
exception-handling experience that builds on and enhances Ruby's
exception-handling mechanisms.

Polyphony includes a full-blown HTTP server implementation with integrated
support for HTTP 1 & 2, WebSockets, TLS/SSL termination and more. Polyphony also
provides fiber-aware adapters for connecting to PostgreSQL and Redis servers.
More adapters are being actively developed.

### Features
{: .no_toc }

- Co-operative scheduling of concurrent tasks using Ruby fibers.
- High-performance event reactor for handling I/O, timer, and other events.
- Natural, sequential programming style that makes it easy to reason about
  concurrent code.
- Abstractions and constructs for controlling the execution of concurrent code:
  supervisors, cancel scopes, throttling, resource pools etc.
- Code can use native networking classes and libraries, growing support for
  third-party gems such as pg and redis.
- Use stdlib classes such as TCPServer and TCPSocket and Net::HTTP.
- Impressive performance and scalability characteristics, in terms of both
  throughput and memory consumption (see below)

## Taking Polyphony for a Spin

Polyphony is different from other reactor-based solutions for Ruby in that
there's no need to use special classes for building your app, and there's no
need to setup reactor loops. Everything works the same except you can perform
multiple operations at the same time by creating fibers. In order to start a new
concurrent operation, you simply use `Kernel#spin`, which spins up a new fiber
and schedules it for running:

```ruby
require 'polyphony'

# Kernel#spin returns a Fiber instance
counter = spin do
  count = 1
  loop do
    sleep 1
    puts "count: #{count}"
    count += 1
  end
end

puts "Press return to stop this program"
gets
```

The above program spins up a fiber named `counter`, which counts to infinity.
Meanwhile, the *main* fiber waits for input from the user, and then exits.
Notice how we haven't introduced any custom classes, and how we used stock APIs
such as `Kernel#sleep` and `Kernel#gets`. The only hint that this program is
concurrent is the call to `Kernel#spin`.

Behind the scenes, Polyphony takes care of automatically switching between
fibers, letting each fiber advance at its own pace according to its duties. For
example, when the main fiber calls `gets`, Polyphony starts waiting for data to
come in on `STDIN` and then switches control to the `counter` fiber. When the
`counter` fiber calls `sleep 1`, Polyphony starts a timer, and goes looking for
other work. If no other fiber is ready to run, Polyphony simply waits for at
least one event to occur, and then resumes the corresponding fiber.

## Fibers vs Threads

Most Ruby developers are familiar with threads, but fibers remain a little
explored and little understood concept in the Ruby language. While A thread is
an OS abstraction that is controlled by the OS, a fiber represents an execution
context that can be paused and resumed by the application, and has no
counterpart at the OS level.

When used for writing concurrent programming, fibers offer multiple benefits
over threads. They consume much less RAM than threads, and switching between
them is faster than switching between threads. In addition, since fibers require
no cooperation from the OS, an application can create literally millions of them
given enough RAM. Those advantages make fibers a compelling solution for creating
pervasively concurrent applications, even when using a dynamic high-level "slow"
language such as Ruby.

Ruby programs will only partly benefit from using mutiple threads for processing
work loads (due to the GVL), but fibers are a great match mostly for programs
that are I/O bound (that means spending most of their time talking to the
outside world). A fiber-based web-server, for example, can juggle thousands of
active concurrent connections, each advancing at its own pace, consuming only a
single CPU core.

Nevertheless, Polyphony fully supports multithreading, with each thread having
its own fiber run queue and its own libev event loop. In addition, Polyphony
enables cross-thread communication using  

## Fibers vs Callbacks

Programming environments such as Node.js and libraries such as EventMachine have
popularized the usage of event loops for achieving concurrency. The application
is wrapped in a loop that polls for events and fires application-provided
callbacks that act on those events - for example receiving data on a socket
connection, or waiting for a timer to elapse.

While these callback-based solutions are established technologies and are used
frequently to build concurrent apps, they do have some major drawbacks. Firstly,
they force the developer to split the business logic into small pieces, each
being ran inside of a callback. Secondly, they complicate state management,
because state associated with the business logic cannot be kept *with* the
business logic, it has to be stored elsewhere. Finally, callback-based
concurrency complicates debugging, since a stacktrace at any given point in time
will always originate in the event loop, and will not contain any information on
the chain of events leading to the present moment.

Fibers, in contrast, let the developer express the business logic in a
sequential, easy to read manner: do this, then that. State can be stored right
in the business logic, as local variables. And finally, the sequential
programming style makes it much easier to debug your code, since stack traces
contain the entire history of execution from the app's inception. 

## Structured Concurrency

Polyphony's tagline is "fine-grained concurrency for Ruby", because it makes it
really easy to spin up literally thousands of fibers that perform concurrent
work. But running such a large number of concurrent operations also means you
need tools for managing all that concurrency.

For that purpose, Polyphony follows a paradigm called *structured concurrency*.
The basic idea behind structured concurrency is that fibers are organised in a
hierarchy starting from the main fiber. A fiber spun by any given fiber is
considered a child of that fiber, and its lifetime is guaranteed to be limited
to that of its parent fiber. That is why in the example above, the `counter`
fiber is automatically stopped when the main fiber stops running.

The same goes for exception handling. Whenever an error occurs, if no suitable
`rescue` block has been defined for the fiber in which the exception was raised,
the exception will bubble up through the fiber's parent, grandparent etc, until
the exception is handled, up to the main fiber. If the exception was not
handled, the program will exit and dump the exception information just like a
normal Ruby program.

## Controlling Fiber Execution

Polyphony offers a wide range of APIs for controlling fibers that make it easy
to prevent your program turning into an incontrollable concurrent mess. In order
to control fibers, Polyphony introduces various APIs for stopping fibers,
scheduling fibers, awaiting for fibers to terminate, and even restarting them:

```ruby
f = spin do
  puts "going to sleep"
  sleep 1
  puts "done sleeping"
ensure
  puts "stopped"
end

sleep 0.5
f.stop
f.restart
f.await
```

The output of the above program will be:

```
going to sleep
stopped
going to sleep
done sleeping
stopped
```

The `Fiber#await` method waits for a fiber to terminate, and returns the fiber's
return value:

```ruby
a = spin { sleep 1; :foo }
b = spin { a.await }
b.await #=> :foo
```

In the program above the main fiber waits for fiber `b` to terminate, and `b`
waits for fiber `a` to terminate. The return value of `a.await` is `:foo`, and
hence the return value of `b.await` is also `foo`.

If we need to wait for multiple fibers, we can use `Fiber::await` or
`Fiber::select`:

```ruby
# get result of a bunch of fibers
fibers = 3.times.map { |i| spin { i * 10 } }
Fiber.await(*fibers) #=> [0, 10, 20]

# get the fastest reply of a bunch of URLs
fibers = urls.map { |u| spin { [u, HTTParty.get(u)] } }
# Fiber.select returns an array containing the fiber and its result
Fiber.select(*fibers) #=> [fiber, [url, result]]
```

Finally, fibers can be supervised, in a similar manner to Erlang supervision
trees. The `Kernel#supervise` method will wait for all child fibers to terminate
before returning, and can optionally restart any child fiber that has terminated
normally or with an exception:

```ruby
fiber1 = spin { sleep 1; raise 'foo' }
fiber2 = spin { sleep 1 }

supervise # blocks and then propagates the error raised in fiber1 
```

## Message Passing

Polyphony also provides a comprehensive solution for using fibers as actors, in
a similar fashion to Erlang processes. Fibers can exchange messages between each
other, allowing each part of a concurrent system to function in a completely
autonomous manner. For example, a chat application can encapsulate each chat
room in a completely self-contained fiber:

```ruby
def chat_room
  subscribers = []

  loop do
    # receive waits for a message to come in
    case receive
    # Using Ruby 2.7's pattern matching
    in [:subscribe, subscriber]
      subscribers << subscriber
    in [:unsubscribe, subscriber]
      subscribers.delete subscriber
    in [:add_message, name, message]
      subscribers.each { |s| s.call(name, message) }
    end
  end
end

CHAT_ROOMS = Hash.new do |h, n|
  h[n] = spin { chat_room }
end
```

Notice how the state (the `subscribers` variable) stays local, and how the logic
of the chat room is expressed in a way that is both compact and easy to extend.
Also notice how the chat room is written as an infinite loop. This is a common
pattern in Polyphony, since fibers can always be stopped at any moment. 

The code for handling a chat room user might be expressed as follows:

```ruby
def chat_user_handler(user_name, connection)
  room = nil
  message_subscriber = proc do |name, message|
    connection.puts "#{name}: #{message}"
  end
  while command = connection.gets
    case command
    when /^connect (.+)/
      room&.send [:unsubscribe, message_subscriber]
      room = CHAT_ROOMS[$1]
    when "disconnect"
      room&.send [:unsubscribe, message_subscriber]
      room = nil
    when /^send (.+)/
      room&.send [:add_message, user_name, $1]
    end
  end
end
```

## Other Concurrency Constructs

Polyphony includes various constructs that complement fibers. Resource pools
provide a generic solution for controlling concurrent access to limited
resources, such as database connections. A resource pool assures only one fiber
has access to a given resource at any time:

```ruby
DB_CONNECTIONS = Polyphony::ResourcePool.new(limit: 5) do
  PG.connect(DB_OPTS)
end

def query_records(sql)
  DB_CONNECTIONS.acquire do |db|
    db.query(sql).to_a
  end
end
```

Throttlers can be useful for rate limiting, for example preventing blacklisting
your system in case it sends too many emails, even across fibers:

```ruby
MAX_EMAIL_RATE = 10 # max. 10 emails per second
EMAIL_THROTTLER = Polyphony::Throttler.new(MAX_EMAIL_RATE)

def send_email(addr, content)
  EMAIL_THROTTLER.process do
    ...
  end
end
```

In addition, various global methods (defined on the `Kernel` module) provide
common functionality, such as using timeouts:

```ruby
# perform an delayed action (in a separate fiber)
after(10) { notify_user }

# perform a recurring action with time drift correction
every(1) { p Time.now }

# perform an operation with timeout without raising an exception
move_on_after(10) { perform_query }

# perform an operation with timeout, raising a Polyphony::Cancel exception
cancel_after(10) { perform_query }
```

## The System Agent

In order to implement automatic fiber switching when performing blocking
operations, Polyphony introduces a concept called the *system agent*. The system
agent is an object having a uniform interface, that performs all blocking
operations.

While a standard event loop-based solution would implement a blocking call
separately from the fiber scheduling, the system agent integrates the two to
create a blocking call that is already knows how to switch and schedule fibers.
For example, in Polyphony all APIs having to do with reading from files or
sockets end up calling `Thread.current.agent.read`, which does all the work.

This design offers some major advantages over other designs. It minimizes memory
allocations, of both Ruby objects and C structures. For example, instead of
having to allocate libev watchers on the heap and then pass them around, they
are allocated on the stack instead, which saves up on both memory and CPU cycles.

In addition, the agent interface includes two methods that allow maximizing
server performance by accepting connections and reading from sockets in a tight
loop. Here's a naive implementation of an HTTP/1 server:

```ruby
require 'http/parser'
require 'polyphony'

def handle_client(socket)
  parser = Http::Parser.new
  reqs = []
  parser.on_message_complete = proc { |env| reqs << { foo: :bar } }

  Thread.current.agent.read_loop(socket) do |data|
    parser << data
    reqs.each { |r| reply(socket, r) }
    reqs.clear
  end
end

def reply(socket)
  data = "Hello world!\n"
  headers = "Content-Type: text/plain\r\nContent-Length: #{data.bytesize}\r\n"
  socket.write "HTTP/1.1 200 OK\r\n#{headers}\r\n#{data}"
end

server = TCPServer.open('0.0.0.0', 1234)
puts "listening on port 1234"

Thread.current.agent.accept_loop(server) do |client|
  spin { handle_client(client) }
end
```

The `#read_loop` and `#accept_loop` agent methods implement tight loops that
provide a significant boost to performance (up to +30% better throughput.)

Currently, Polyphony includes a single system agent based on
[libev](http://pod.tst.eu/http://cvs.schmorp.de/libev/ev.pod). In the future,
Polyphony will include other platform-specific system agents, such as a Windows
agent using
[IOCP](https://docs.microsoft.com/en-us/windows/win32/fileio/i-o-completion-ports),
or an [io_uring](https://unixism.net/loti/what_is_io_uring.html) agent,
which might be a game-changer for writing highly-concurrent Ruby-based web apps.

## Writing Web Apps with Polyphony

Polyphony includes a full-featured web server implementation that supports
HTTP/1, HTTP/2, and WebSockets, can perform SSL termination (with automatic ALPN
protocol selection), and has preliminary support for Rack (the de-facto standard
Ruby web app interface).

The Polyphony HTTP server has a unique design that calls the application's
request handler after all request headers have been received. This allows the
application to better deal with slow client attacks, big file uploads, and also
to minimize costly memory allocation and GC'ing.

Preliminary benchmarks show the Polyphony web server to be about 3X as fast as
Puma and 20X as fast as Unicorn. With SSL termination and using HTTP 2, the
Polyphony web server is about 2X as fast as Falcon.

### HTTP/1

|Server|requests/sec|average latency|max latency|
|------|-----------:|--------------:|----------:|
|Puma|
|Unicorn|
|Agoo|
|Polyphony|

### HTTP/2 with SSL Termination

|Server|requests/sec|average latency|max latency|
|------|-----------:|--------------:|----------:|
|Falcon|
|Polyphony|

(*Non-official benchmark with a basic "Hello world" Rack application. The usual
caveats regarding benchmarks should be applied here.*)

## Integrating Polyphony with other Gems

Polyphony aims to be a comprehensive concurrency solution for Ruby, and to
enable developers to use a maximum of core and stdlib APIs transparently in a
multi-fiber envrionment. Polyphony also provides adapters for common gems such
as postgres and redis, allowing using those gems in a fiber-aware manner.

For gems that do not yet have a fiber-aware adapter, Polyphony offers a general
solution in the form of a thread pool. A thread pool lets you offload blocking
method calls (that block the entire thread) onto worker threads, letting you
continue with other work while waiting for the call to return. For example,
here's how an `sqlite` adapter might work:

```ruby
class SQLite3::Database
  THREAD_POOL = Polyphony::ThreadPool.new

  alias_method :orig_execute, :execute
  def execute(sql, *args)
    THREAD_POOL.process { orig_execute(sql, *args) }
  end
end
```

Other cases might require converting a callback-based interface into a blocking
fiber-aware one. Here's (a simplified version of) how Polyphony uses the
callback-based `http_parser.rb` gem to parse incoming HTTP/1 requests:

```ruby
class HTTP1Adapter
  ...

  def on_headers_complete(headers)
    @pending_requests << Request.new(headers, self)
  end

  def each(&block)
    while (data = @connection.readpartial(8192))
      # feed parser
      @parser << data
      while (request = @pending_requests.shift)
        block.call(request)
        return unless request.keep_alive?
      end
    end
  end

  ...
end
```

In the code snippet above, the solution is quite simple. The fiber handling the
connection loops waiting for data to be read from the socket. Once the data
arrives, it is fed to the HTTP parser. The HTTP parser will call the
`on_headers_complete` callback, which simply adds a request to the requests
queue. The code then continues to handle any requests still in the queue.

## Future Directions

Polyphony is a young project, and will still need a lot of development effort to
reach version 1.0. Here are some of the exciting directions we're working on.

- Support for more core and stdlib APIs
- More adapters for gems with C-extensions, such as `mysql`, `sqlite3` etc
- Use `io_uring` agent as alternative to the libev agent 
- More concurrency constructs for building highly concurrent applications