README.md in field_test-0.2.4 vs README.md in field_test-0.3.0
- old
+ new
@@ -1,15 +1,17 @@
# Field Test
:maple_leaf: A/B testing for Rails
- Designed for web and email
-- Comes with a [handy dashboard](https://fieldtest.dokkuapp.com/)
+- Comes with a [dashboard](https://fieldtest.dokkuapp.com/) to view results and update variants
- Seamlessly handles the transition from anonymous visitor to logged in user
Uses [Bayesian statistics](https://www.evanmiller.org/bayesian-ab-testing.html) to evaluate results so you don’t need to choose a sample size ahead of time.
+[](https://travis-ci.org/ankane/field_test)
+
## Installation
Add this line to your application’s Gemfile:
```ruby
@@ -17,20 +19,21 @@
```
Run:
```sh
-rails g field_test:install
+rails generate field_test:install
+rails db:migrate
```
And mount the dashboard in your `config/routes.rb`:
```ruby
mount FieldTest::Engine, at: "field_test"
```
-Be sure to [secure the dashboard](#security) in production.
+Be sure to [secure the dashboard](#dashboard-security) in production.
## Getting Started
@@ -43,16 +46,22 @@
- red
- green
- blue
```
-Refer to it in views, controllers, and mailers.
+Refer to it in controllers, views, and mailers.
```ruby
button_color = field_test(:button_color)
```
+To make testing easier, you can specify a variant with query parameters
+
+```
+http://localhost:3000/?field_test[button_color]=green
+```
+
When someone converts, record it with:
```ruby
field_test_converted(:button_color)
```
@@ -65,27 +74,65 @@
winner: green
```
All calls to `field_test` will now return the winner, and metrics will stop being recorded.
-## Features
+You can get the list of experiments and variants for a user with:
-You can specify a variant with query parameters to make testing easier
+```ruby
+field_test_experiments
+```
+## JavaScript and Native Apps
+
+For JavaScript and native apps, add calls to your normal endpoints.
+
+```ruby
+class CheckoutController < ActionController::API
+ def start
+ render json: {button_color: field_test(:button_color)}
+ end
+
+ def finish
+ field_test_converted(:button_color)
+ # ...
+ end
+end
```
-?field_test[button_color]=green
+
+For anonymous visitors in native apps, pass a `Field-Test-Visitor` header with a unique identifier.
+
+## Participants
+
+Any model or string can be a participant in an experiment.
+
+For web requests, it uses `current_user` (if it exists) and an anonymous visitor id to determine the participant. Set your own with:
+
+```ruby
+class ApplicationController < ActionController::Base
+ def field_test_participant
+ current_company
+ end
+end
```
-Assign a specific variant to a user with:
+For mailers, it tries `@user` then `params[:user]` to determine the participant. Set your own with:
```ruby
-experiment = FieldTest::Experiment.find(:button_color)
-experiment.variant(participant, variant: "green")
+class ApplicationMailer < ActionMailer::Base
+ def field_test_participant
+ @company
+ end
+end
```
-You can also change a user’s variant from the dashboard.
+You can also manually pass a participant with:
+```ruby
+field_test(:button_color, participant: company)
+```
+
## Config
By default, bots are returned the first variant and excluded from metrics. Change this with:
```yml
@@ -124,24 +171,39 @@
weights:
- 85
- 15
```
+To help with GDPR compliance, you can switch from cookies to [anonymity sets](https://privacypatterns.org/patterns/Anonymity-set) for anonymous visitors. Visitors with the same IP mask and user agent are grouped together.
+
+```yml
+cookies: false
+```
+
+## Dashboard Config
+
If the dashboard gets slow, you can make it faster with:
```yml
cache: true
```
This will use the Rails cache to speed up winning probability calculations.
-## Funnels
+If you need more precision, set:
+```yml
+precision: 1
+```
+
+## Multiple Goals
+
You can set multiple goals for an experiment to track conversions at different parts of the funnel. First, run:
```sh
-rails g field_test:events
+rails generate field_test:events
+rails db:migrate
```
And add to your config:
```yml
@@ -160,24 +222,34 @@
The results for all goals will appear on the dashboard.
## Analytics Platforms
-You can also send experiment data to analytics platforms like [Google Analytics](https://www.google.com/analytics/), [Mixpanel](https://mixpanel.com/), and [Ahoy](https://github.com/ankane/ahoy). Use:
+You may also want to send experiment data as properties to other analytics platforms like [Segment](https://segment.com), [Amplitude](https://amplitude.com), and [Ahoy](https://github.com/ankane/ahoy). Get the list of experiments and variants with:
```ruby
field_test_experiments
```
-to get all experiments and variants for a participant and pass them as properties.
+### Ahoy
-## Security
+You can configure Field Test to use Ahoy’s visitor token instead of creating its own:
+```ruby
+class ApplicationController < ActionController::Base
+ def field_test_participant
+ [ahoy.user, ahoy.visitor_token]
+ end
+end
+```
+
+## Dashboard Security
+
#### Devise
```ruby
-authenticate :user, -> (user) { user.admin? } do
+authenticate :user, ->(user) { user.admin? } do
mount FieldTest::Engine, at: "field_test"
end
```
#### Basic Authentication
@@ -187,16 +259,107 @@
```ruby
ENV["FIELD_TEST_USERNAME"] = "moonrise"
ENV["FIELD_TEST_PASSWORD"] = "kingdom"
```
-## Credits
+## Reference
-A huge thanks to [Evan Miller](https://www.evanmiller.org/) for deriving the Bayesian formulas.
+Assign a specific variant to a user with:
-## TODO
+```ruby
+experiment = FieldTest::Experiment.find(:button_color)
+experiment.variant(participant, variant: "green")
+```
-- Code samples for analytics platforms
+You can also change a user’s variant from the dashboard.
+
+To associate models with field test memberships, use:
+
+```ruby
+class User < ApplicationRecord
+ has_many :field_test_memberships, class_name: "FieldTest::Membership", as: :participant
+end
+```
+
+Now you can do:
+
+```ruby
+user.field_test_memberships
+```
+
+## Upgrading
+
+### 0.3.0
+
+Upgrade the gem and add to `config/field_test.yml`:
+
+```yml
+legacy_participants: true
+```
+
+Also, if you use Field Test in emails, know that the default way participants are determined has changed. Restore the previous way with:
+
+```ruby
+class ApplicationMailer < ActionMailer::Base
+ def field_test_participant
+ message.to.first
+ end
+end
+```
+
+We also recommend upgrading participants when you have time.
+
+#### Upgrading Participants
+
+Field Test 0.3.0 splits the `field_test_memberships.participant` column into `participant_type` and `participant_id`.
+
+To upgrade without downtime, create a migration:
+
+```sh
+rails generate migration upgrade_field_test_participants
+```
+
+with:
+
+```ruby
+class UpgradeFieldTestParticipants < ActiveRecord::Migration[5.2]
+ def change
+ add_column :field_test_memberships, :participant_type, :string
+ add_column :field_test_memberships, :participant_id, :string
+
+ add_index :field_test_memberships, [:participant_type, :participant_id, :experiment],
+ unique: true, name: "index_field_test_memberships_on_participant_and_experiment"
+ end
+end
+```
+
+After you run it, writes will go to both the old and new sets of columns.
+
+Next, backfill data:
+
+```ruby
+FieldTest::Membership.where(participant_id: nil).find_each do |membership|
+ participant = membership.participant
+
+ if participant.include?(":")
+ participant_type, _, participant_id = participant.rpartition(":")
+ participant_type = nil if participant_type == "cookie" # legacy
+ else
+ participant_id = participant
+ end
+
+ membership.update!(
+ participant_type: participant_type,
+ participant_id: participant_id
+ )
+end
+```
+
+Finally, remove `legacy_participants: true` from the config file. Once you confirm it’s working, you can drop the `participant` column (you can rename it first just to be extra safe).
+
+## Credits
+
+A huge thanks to [Evan Miller](https://www.evanmiller.org/) for deriving the Bayesian formulas.
## History
View the [changelog](https://github.com/ankane/field_test/blob/master/CHANGELOG.md)