'
body "There were #{step_responses['keen']} events in the last 24 hours!"
end
end
```
Now modify the example to use your specific information. You'll want to specify a `to` and a `from` address if you're using Sendgrid, and a `to` phone number if you're using Twilio. Everything you need to change is marked with `<>`. You'll also want to remove either Sendgrid or Twilio blocks you're not using them.
Save the file and test this job using the same `jobs:run_once` rake task that we used before.
``` shell
$ foreman run rake jobs:run_once[jobs/first_job.rb]
```
The output of each step will be logged to the console, and if everything worked you'll receive an email or a text message within a few seconds!
**Next steps**
+ Write and test more jobs. See the [Pushpop API Documentation](#pushpop-api-documentation) below for more examples of what you can do.
+ Continue on to the deploy guide to deploy a Pushpop instance and start getting regular reports and alerts.
## Deploy Guide
##### Heroku
These instructions are for Heroku, but should be relevant to most environments.
**Prerequisites**
You'll need a Heroku account, and the [Heroku toolbelt](https://toolbelt.heroku.com/) installed.
**Create a new Heroku app**
Make sure you're inside the Pushpop directory.
``` shell
$ heroku create
```
**Commit changes**
If you created a new job from the Quickstart guide, you'll want to commit that code before deploying.
``` shell
$ git commit -am 'Created my first Pushpop job'
```
**Set Heroku config variables**
The easiest way to do this is with the [heroku-config](https://github.com/ddollar/heroku-config) plugin. This step assumes you have created a `.env` file containing your keys as demonstrated in the Quickstart guide.
``` shell
$ heroku plugins:install git://github.com/ddollar/heroku-config.git
$ heroku config:push
```
**Deploy code to Heroku**
Now that your code is commited and config variables pushed we can begin a deploy.
``` shell
$ git push heroku master
```
**Tail logs to confirm it's working**
To see that jobs are running and that there are no errors, tail the logs on Heroku.
``` shell
$ heroku logs --tail
```
Note that if you have jobs that are set to run at specific times of day you might not see output for a while.
Also note - by default this will run all jobs in the `jobs` folder. You might want to delete the `example_job.rb` file in
a separate commit once you've got the hang of things.
##### Other environments
Pushpop is run entirely by one long-running Ruby process. Anywhere you can run this process in a monitored fashion you can run a Pushpop instance. Here's the command:
``` shell
$ foreman run rake jobs:run
```
If you don't want to use foreman and prefer to set the environment variables yourself then all you need is this:
``` shell
$ bundle exec rake jobs:run
```
## Rake Tasks
All `jobs:*` rake tasks optionally take a single filename as a parameter. The file is meant to contain one or more Pushpop jobs. If no filename is specified, all jobs in the jobs folder are considered.
Specifying a specific file looks like this:
``` shell
$ foreman run rake jobs:run[jobs/just_this_job.rb]
```
Here's a list of the available rake tasks:
+ `jobs:describe` - Print out the names of jobs in the jobs folder.
+ `jobs:run_once` - Run each job once, right now.
+ `jobs:run` - Run jobs as scheduled in a long-running process. This is the task used when you deploy.
+ `spec` - Run the specs.
## Pushpop API Documentation
Steps and jobs are the heart of the Pushpop workflow. Ruby jobs files contain one or more jobs, and each job consists of one or more steps.
#### Jobs
Jobs have the following attributes:
+ `name`: (optional) something that describe the job, useful in logs
+ `every_duration`: the frequency at which to run the job
+ `every_options` (optional): options related to when the job runs
+ `steps`: the ordered list of steps to run
These attributes are easily specified using the DSL's block syntax. Here's an example:
``` ruby
job 'print job' do
every 5.minutes
step do
puts "5 minutes later..."
end
end
```
Inside of a `job` configuration block, steps are added by using the `step` method. They can also be
added by using a method registered by a plugin, like `keen` or `twilio`. For more information, see [Plugin Documentation](#plugin-documentation).
The frequency of the job is set via the `every` method. This is basically a passthrough to Clockwork.
Here are some cool things you can do with regard to scheduling:
``` ruby
every 5.seconds
every 24.hours, at: '12:00'
every 24.hours, at: ['00:00', '12:00']
every 24.hours, at: '**:05'
every 24.hours, at: '00:00', tz: 'UTC'
every 5.seconds, at: '10:**'
every 1.week, at: 'Monday 12:30'
```
See the full set of options on the [Clockwork README](https://github.com/tomykaira/clockwork#event-parameters).
##### Job workflow
When a job kicks off, steps are run serially in the order they are specified. Each step is invoked with 2
arguments - the response of the step immediately preceding it, and a map of all responses so far.
The map is keyed by step name, which defaults to a plugin name if a plugin was used but a step name not specified.
Here's an example that shows how the response chain works:
``` ruby
job do
every 5.minutes
step 'one' do
1
end
step 'two' do |response|
5 + response
end
step 'add previous steps' do |response, step_responses|
puts response # prints 5
puts step_responses['one'] + step_responses['two'] # prints 6
end
end
```
If a `step` returns false, subsequent steps **are not run**. Here's a simple example that illustrates this:
``` ruby
job 'lame job' do
every 5.minutes
step 'one' do
false
end
step 'two' do
# never called!
end
end
```
This behavior is designed to make *conditional* alerting easy. Here's an example of a job that only sends an alert
for certain query responses:
``` ruby
job do
every 1.minute
keen do
event_collection 'errors'
analysis_type 'count'
timeframe 'last_1_minute'
end
step 'notify only if there are errors' do |response|
response > 0
end
twilio do |step_responses|
to '+18005555555'
body "There were #{step_responses['keen']} errors in the last minute!"
end
end
```
In this example, the `twilio` step will only be ran if the `keen` step returned a count greater than 0.
#### Steps
Steps have the following attributes:
+ `name`: (optional) something that describes the step. Useful in logs, and is the key in the `step_responses` hash. Defaults to plugin name if a plugin is used
+ `plugin`: (optional) if the step is backed by a plugin, it's the name of the plugin
+ `block`: A block that runs to configure the step (when a plugin is used) or run it
Steps can be pure Ruby code or leverage a plugin DSL. Plugins are just fancy abstractions for creating steps.
Steps have built-in support for ERB templating. This is useful for generating more complex emails and reports.
Here's an example that uses a template:
``` ruby
sendgrid do |response, step_responses|
to 'josh+pushpop@keen.io'
from 'pushpopapp+123@keen.io'
subject 'Pingpong Daily Response Time Report'
body template 'pingpong_report.html.erb', response, step_responses
preview false
end
```
`template` is a function that renders a template in context of the step responses and returns a string.
The first argument is a template file name, located in the `templates` directory by default.
The second and third arguments are the `response` and `step_responses` respectively.
An optional fourth parameter can be used to change the path templates are looked up in.
Here's a very simple template:
``` erb
Daily Report
We got <%= response %> new users today!
```
## Recipes
The community-driven [pushpop-recipes](https://github.com/keenlabs/pushpop-recipes) repository contains jobs and templates
for doing common things with Pushpop. Check it out for some inspiration!
## Plugin Documentation
Plugins are located at `lib/plugins`. They are loaded automatically.
##### Keen
The `keen` plugin gives you a DSL to specify Keen query parameters. When it runs, it
passes those parameters to the [keen gem](https://github.com/keenlabs/keen-gem), which
in turn runs the query against the Keen IO API.
Here's an example that shows most of the options you can specify:
``` ruby
job 'daily average response time by check for successful requests in april' do
keen do
event_collection 'checks'
analysis_type 'average'
target_property 'request.duration'
group_by 'check.name'
interval 'daily'
timeframe ({ start: '2014-04-01T00:00Z' })
filters [{ property_name: "response.successful",
operator: "eq",
property_value: true }]
end
end
```
The `keen` plugin requires that the following environment variables are set: `KEEN_PROJECT_ID` and `KEEN_READ_KEY`.
A `steps` method is also supported for [funnels](https://keen.io/docs/data-analysis/funnels/),
as well as `analyses` for doing a [multi-analysis](https://keen.io/docs/data-analysis/multi-analysis/).
##### Sendgrid
The `sendgrid` plugin gives you a DSL to specify email parameters like subject and body.
Here's an example:
``` ruby
job 'send an email' do
sendgrid do
to 'josh+pushpop@keen.io'
from 'pushpopapp+123@keen.io'
subject 'Is your inbox lonely?'
body 'This email was intentionally left blank.'
preview false
end
end
```
The `sendgrid` plugin requires that the following environment variables are set: `SENDGRID_DOMAIN`, `SENDGRID_USERNAME`, and `SENDGRID_PASSWORD`.
The `preview` directive is optional and defaults to false. If you set it to true, the email contents will print out
to the console, but the email will not be sent.
The `body` method can take a string, or it can take the same parameters as `template`,
in which case it will render a template to create the body. For example:
``` ruby
body 'pingpong_report.html.erb', response, step_responses
```
##### Twilio
The `twilio` plugin provides DSL to specify SMS recipient information as well as the text itself.
Here's an example:
``` ruby
job 'send a text' do
twilio do
to '18005555555'
body 'Breathe in through the nose, out through the mouth.'
end
end
```
The `twilio` plugin requires that the following environment variables are set: `TWILIO_AUTH_TOKEN`, `TWILIO_SID`, and `TWILIO_FROM`.
## Creating plugins
Plugins are just subclasses of `Pushpop::Step`. Plugins should implement a run method, and
register themselves. Here's a simple plugin that stops job execution if the input into the step is 0:
``` ruby
module Pushpop
class BreakIfZero < Step
PLUGIN_NAME = 'break_if_zero'
def run(last_response=nil, step_responses=nil)
last_response == 0
end
end
Pushpop::Job.register_plugin(BreakIfZero::PLUGIN_NAME, BreakIfZero)
end
# now in your job you can use the break_if_zero step
job do
step do [0, 1].shuffle.first end
break_if_zero
step do puts 'made it through!' end
end
```
## Usage as a Ruby gem
Pushpop is also a Ruby gem, so you can add it to an existing Ruby project. Here's how to do that:
**Install the gem**
``` ruby
# bundler
gem 'pushpop'
# not bundler
gem install 'pushpop'
```
**Require job files and run**
Once the gem is available, you should be able to load in Pushpop job files. Once a file loads,
all of its jobs are ready to be run or scheduled.
``` ruby
load 'some_job.rb'
# run the jobs once
Pushpop.run
# or schedule and run the jobs with clockwork
Pushpop.schedule
Clockwork.manager.run
```
Note that `pushpop` does not declare dependencies other than `clockwork` and `keen`. If you're using
Pushpop plugins like Sendgrid or Twilio you'll need to install and require those dependencies explicitly.
## Contributing
Issues and pull requests are welcome!
**Wishlist**
+ Add plugins for more data collection and notification services
+ Add a web interface that shows the job names, job results, and a countdown to the next run
+ Add a web interface that lets you preview emails in the browser
+ Beautiful email templates with support for typical Keen IO query responses (groups, series, etc)
**Testing**
Pushpop has a full set of specs (including plugins). Run them like this:
``` shell
$ bundle exec rake spec
```
