README.md in tobox-0.3.2 vs README.md in tobox-0.4.0
- old
+ new
@@ -18,10 +18,11 @@
- [Inbox](#inbox)
- [Plugins](#plugins)
- [Zeitwerk](#zeitwerk)
- [Sentry](#sentry)
- [Datadog](#datadog)
+ - [Stats](#stats)
- [Supported Rubies](#supported-rubies)
- [Rails support](#rails-support)
- [Why?](#why)
- [Development](#development)
- [Contributing](#contributing)
@@ -478,9 +479,72 @@
c.tracing.instrument :tobox
end
# tobox.rb
plugin(:datadog)
+```
+
+<a id="markdown-datadog" name="stats"></a>
+### Stats
+
+The `stats` plugin collects statistics related with the outbox table periodically, and exposes them to app code (which can then relay them to a statsD collector, or similar tool).
+
+```ruby
+plugin(:stats)
+on_stats(5) do |stats_collector| # every 5 seconds
+ stats = stats_collector.collect
+ #
+ # stats => {
+ # pending_count: number of new events in the outbox table
+ # failing_count: number of events which have failed processing but haven't reached the threshold
+ # failed_count: number of events which have failed the max number of tries
+ # inbox_count: (if used) number of events marked as received in the inbox table
+ # }
+ #
+ # now you can send them to your statsd collector
+ #
+ StatsD.gauge('outbox_pending_backlog', stats[:pending_count])
+end
+```
+
+#### Bring your own leader election
+
+The stats collection runs on every `tobox` initiated. If you're launching it in multiple servers / containers / pods, this means you'll be collecting statistics about the same database on all of these instances. This may not be desirable, and you may want to do this collection in a single instance. This is not a problem that `tobox` can solve by itself, so you'll have to take care of that yourself. Still, here are some cheap recommendations.
+
+##### Postgres advisory locks
+
+If your database is PostgreSQL, you can leverage session-level [advisory locks](https://www.postgresql.org/docs/current/explicit-locking.html#ADVISORY-LOCKS) to ensure single-instance access to this functionality. `tobox` also exposes the database instance to the `on_stats` callback:
+
+```ruby
+c.on_stats(5) do |stats_collector, db|
+ if db.get(Sequel.function(:pg_try_advisory_lock, 1))
+ stats = stats_collector.collect
+ StatsD.gauge('outbox_pending_backlog', stats[:pending_count])
+ end
+end
+```
+
+If a server goes down, one of the remaining ones will acquire the lock and ensure stats processing.
+
+##### Redis distributed locks
+
+If you're already using [redis](https://redis.io/), you can use its distributed lock feature to achieve the goal:
+
+```ruby
+# using redlock
+c.on_stats(5) do |stats_collector, db|
+ begin
+ lock_info = lock_manager.lock("outbox", 5000)
+
+ stats = stats_collector.collect
+ StatsD.gauge('outbox_pending_backlog', stats[:pending_count])
+
+ # extend to hold the lock for the next loop
+ lock_info = lock_manager.lock("outbox", 5000, extend: lock_info)
+ rescue Redlock::LockError
+ # some other server already has the lock, try later
+ end
+end
```
<a id="markdown-supported-rubies" name="supported-rubies"></a>
## Supported Rubies