README.md in strong_migrations-0.3.1 vs README.md in strong_migrations-0.4.0
- old
+ new
@@ -14,55 +14,92 @@
gem 'strong_migrations'
```
## How It Works
-Strong Migrations detects potentially dangerous operations in migrations, prevents them from running by default, and provides instructions on safer ways to do what you want. Here’s an example:
+Strong Migrations detects potentially dangerous operations in migrations, prevents them from running by default, and provides instructions on safer ways to do what you want.
-```
-=== Dangerous operation detected #strong_migrations ===
+
-ActiveRecord caches attributes which causes problems
-when removing columns. Be sure to ignore the column:
+## Dangerous Operations
-class User < ApplicationRecord
- self.ignored_columns = ["some_column"]
-end
+The following operations can cause downtime or errors:
-Deploy the code, then wrap this step in a safety_assured { ... } block.
+- [[+]](#removing-a-column) removing a column
+- [[+]](#adding-a-column-with-a-default-value) adding a column with a non-null default value to an existing table
+- [[+]](#backfilling-data) backfilling data
+- [[+]](#adding-an-index) adding an index non-concurrently
+- [[+]](#adding-a-reference) adding a reference
+- [[+]](#adding-a-foreign-key) adding a foreign key
+- [[+]](#renaming-or-changing-the-type-of-a-column) changing the type of a column
+- [[+]](#renaming-or-changing-the-type-of-a-column) renaming a column
+- [[+]](#renaming-a-table) renaming a table
+- [[+]](#creating-a-table-with-the-force-option) creating a table with the `force` option
+- [[+]](#using-change_column_null-with-a-default-value) using `change_column_null` with a default value
+- [[+]](#adding-a-json-column) adding a `json` column
-class RemoveColumn < ActiveRecord::Migration[5.2]
+Also checks for best practices:
+
+- [[+]](#) keeping non-unique indexes to three columns or less
+
+## The Zero Downtime Way
+
+### Removing a column
+
+#### Bad
+
+ActiveRecord caches database columns at runtime, so if you drop a column, it can cause exceptions until your app reboots.
+
+```ruby
+class RemoveSomeColumnFromUsers < ActiveRecord::Migration[5.2]
def change
- safety_assured { remove_column :users, :some_column }
+ remove_column :users, :some_column
end
end
```
-## Dangerous Operations
+#### Good
-The following operations can cause downtime or errors:
+1. Tell ActiveRecord to ignore the column from its cache
-- adding a column with a non-null default value to an existing table
-- removing a column
-- changing the type of a column
-- setting a `NOT NULL` constraint with a default value
-- renaming a column
-- renaming a table
-- creating a table with the `force` option
-- adding an index non-concurrently (Postgres only)
-- adding a `json` column to an existing table (Postgres only)
+ ```ruby
+ class User < ApplicationRecord
+ self.ignored_columns = ["some_column"]
+ end
+ ```
-Also checks for best practices:
+2. Deploy code
+3. Write a migration to remove the column (wrap in `safety_assured` block)
-- keeping non-unique indexes to three columns or less
+ ```ruby
+ class RemoveSomeColumnFromUsers < ActiveRecord::Migration[5.2]
+ def change
+ safety_assured { remove_column :users, :some_column }
+ end
+ end
+ ```
-## The Zero Downtime Way
+4. Deploy and run migration
### Adding a column with a default value
+#### Bad
+
Adding a column with a non-null default causes the entire table to be rewritten.
+```ruby
+class AddSomeColumnToUsers < ActiveRecord::Migration[5.2]
+ def change
+ add_column :users, :some_column, :text, default: "default_value"
+ end
+end
+```
+
+> This operation is safe in Postgres 11+
+
+#### Good
+
Instead, add the column without a default value, then change the default.
```ruby
class AddSomeColumnToUsers < ActiveRecord::Migration[5.2]
def up
@@ -74,158 +111,351 @@
remove_column :users, :some_column
end
end
```
-Don’t backfill existing rows in this migration, as it can cause downtime. See the next section for how to do it safely.
+See the next section for how to backfill.
-> With Postgres, this operation is safe as of Postgres 11
-
### Backfilling data
-To backfill data, use the Rails console or a separate migration with `disable_ddl_transaction!`. Avoid backfilling in a transaction, especially one that alters a table. See [this great article](https://wework.github.io/data/2015/11/05/add-columns-with-default-values-to-large-tables-in-rails-postgres/) on why.
+#### Bad
+Backfilling in the same transaction that alters a table locks the table for the [duration of the backfill](https://wework.github.io/data/2015/11/05/add-columns-with-default-values-to-large-tables-in-rails-postgres/).
+
```ruby
+class AddSomeColumnToUsers < ActiveRecord::Migration[5.2]
+ def change
+ add_column :users, :some_column, :text
+ User.update_all some_column: "default_value"
+ end
+end
+```
+
+Also, running a single query to update data can cause issues for large tables.
+
+#### Good
+
+There are three keys: batching, throttling, and running it outside a transaction. Use the Rails console or a separate migration with `disable_ddl_transaction!`.
+
+```ruby
class BackfillSomeColumn < ActiveRecord::Migration[5.2]
disable_ddl_transaction!
def change
- # Rails 5+
- User.in_batches.update_all some_column: "default_value"
-
- # Rails < 5
- User.find_in_batches do |records|
- User.where(id: records.map(&:id)).update_all some_column: "default_value"
+ User.in_batches do |relation|
+ relation.update_all some_column: "default_value"
+ sleep(0.1) # throttle
end
end
end
```
-### Removing a column
+### Adding an index
-ActiveRecord caches database columns at runtime, so if you drop a column, it can cause exceptions until your app reboots. To prevent this:
+#### Bad
-1. Tell ActiveRecord to ignore the column from its cache
+In Postgres, adding a non-concurrent indexes lock the table.
- ```ruby
- # For Rails 5+
- class User < ApplicationRecord
- self.ignored_columns = ["some_column"]
+```ruby
+class AddSomeIndexToUsers < ActiveRecord::Migration[5.2]
+ def change
+ add_index :users, :some_column
end
+end
+```
- # For Rails < 5
- class User < ActiveRecord::Base
- def self.columns
- super.reject { |c| c.name == "some_column" }
- end
+#### Good
+
+Add indexes concurrently.
+
+```ruby
+class AddSomeIndexToUsers < ActiveRecord::Migration[5.2]
+ disable_ddl_transaction!
+
+ def change
+ add_index :users, :some_column, algorithm: :concurrently
end
- ```
+end
+```
-2. Deploy code
-3. Write a migration to remove the column (wrap in `safety_assured` block)
+If you forget `disable_ddl_transaction!`, the migration will fail. Also, note that indexes on new tables (those created in the same migration) don’t require this. Check out [gindex](https://github.com/ankane/gindex) to quickly generate index migrations without memorizing the syntax.
- ```ruby
- class RemoveSomeColumnFromUsers < ActiveRecord::Migration[5.2]
- def change
- safety_assured { remove_column :users, :some_column }
+### Adding a reference
+
+#### Bad
+
+Rails adds a non-concurrent index to references by default, which is problematic for Postgres.
+
+```ruby
+class AddReferenceToUsers < ActiveRecord::Migration[5.2]
+ def change
+ add_reference :users, :city
+ end
+end
+```
+
+#### Good
+
+Make sure the index is added concurrently.
+
+```ruby
+class AddReferenceToUsers < ActiveRecord::Migration[5.2]
+ disable_ddl_transaction!
+
+ def change
+ add_reference :users, :city, index: false
+ add_index :users, :city_id, algorithm: :concurrently
+ end
+end
+```
+
+For polymorphic references, add a compound index on type and id.
+
+### Adding a foreign key
+
+#### Bad
+
+In Postgres, new foreign keys are validated by default, which acquires an `AccessExclusiveLock` that can be [expensive on large tables](https://travisofthenorth.com/blog/2017/2/2/postgres-adding-foreign-keys-with-zero-downtime).
+
+```ruby
+class AddForeignKeyOnUsers < ActiveRecord::Migration[5.2]
+ def change
+ add_foreign_key :users, :orders
+ end
+end
+```
+
+#### Good
+
+Instead, validate it in a separate migration with a more agreeable `RowShareLock`. This approach is documented by Postgres to have “[the least impact on other work](https://www.postgresql.org/docs/current/sql-altertable.html).”
+
+For Rails 5.2+, use:
+
+```ruby
+class AddForeignKeyOnUsers < ActiveRecord::Migration[5.2]
+ def change
+ add_foreign_key :users, :orders, validate: false
+ end
+end
+```
+
+Then validate it in a separate migration.
+
+```ruby
+class ValidateForeignKeyOnUsers < ActiveRecord::Migration[5.2]
+ def change
+ validate_foreign_key :users, :orders
+ end
+end
+```
+
+For Rails < 5.2, use:
+
+```ruby
+class AddForeignKeyOnUsers < ActiveRecord::Migration[5.1]
+ def change
+ safety_assured do
+ execute 'ALTER TABLE "users" ADD CONSTRAINT "fk_rails_c1e9b98e31" FOREIGN KEY ("order_id") REFERENCES "orders" ("id") NOT VALID'
end
end
- ```
+end
+```
-4. Deploy and run migration
+Then validate it in a separate migration.
+```ruby
+class ValidateForeignKeyOnUsers < ActiveRecord::Migration[5.1]
+ def change
+ safety_assured do
+ execute 'ALTER TABLE "users" VALIDATE CONSTRAINT "fk_rails_c1e9b98e31"'
+ end
+ end
+end
+```
+
### Renaming or changing the type of a column
+#### Bad
+
+```ruby
+class RenameSomeColumn < ActiveRecord::Migration[5.2]
+ def change
+ rename_column :users, :some_column, :new_name
+ end
+end
+```
+
+or
+
+```ruby
+class ChangeSomeColumnType < ActiveRecord::Migration[5.2]
+ def change
+ change_column :users, :some_column, :new_type
+ end
+end
+```
+
+One exception is changing a `varchar` column to `text`, which is safe in Postgres.
+
+#### Good
+
A safer approach is to:
1. Create a new column
2. Write to both columns
3. Backfill data from the old column to the new column
4. Move reads from the old column to the new column
5. Stop writing to the old column
6. Drop the old column
-One exception is changing a `varchar` column to `text`, which is safe in Postgres 9.1+.
-
### Renaming a table
+#### Bad
+
+```ruby
+class RenameUsersToCustomers < ActiveRecord::Migration[5.2]
+ def change
+ rename_table :users, :customers
+ end
+end
+```
+
+#### Good
+
A safer approach is to:
1. Create a new table
2. Write to both tables
3. Backfill data from the old table to new table
4. Move reads from the old table to the new table
5. Stop writing to the old table
6. Drop the old table
-### Adding an index (Postgres)
+### Creating a table with the `force` option
-Add indexes concurrently.
+#### Bad
+The `force` option can drop an existing table.
+
```ruby
-class AddSomeIndexToUsers < ActiveRecord::Migration[5.2]
- disable_ddl_transaction!
+class CreateUsers < ActiveRecord::Migration[5.2]
+ def change
+ create_table :users, force: true do |t|
+ # ...
+ end
+ end
+end
+```
+#### Good
+
+If you intend to drop a table, do it explicitly. Then create the new table without the `force` option:
+
+```ruby
+class CreateUsers < ActiveRecord::Migration[5.2]
def change
- add_index :users, :some_column, algorithm: :concurrently
+ create_table :users do |t|
+ # ...
+ end
end
end
```
-If you forget `disable_ddl_transaction!`, the migration will fail. Also, note that indexes on new tables (those created in the same migration) don’t require this. Check out [gindex](https://github.com/ankane/gindex) to quickly generate index migrations without memorizing the syntax.
+### Using `change_column_null` with a default value
-Rails 5+ adds an index to references by default. To make sure this happens concurrently, use:
+#### Bad
+This generates a single `UPDATE` statement to set the default value.
+
```ruby
-class AddSomeReferenceToUsers < ActiveRecord::Migration[5.2]
- disable_ddl_transaction!
+class ChangeSomeColumnNull < ActiveRecord::Migration[5.2]
+ def change
+ change_column_null :users, :some_column, false, "default_value"
+ end
+end
+```
+#### Good
+
+Backfill the column [safely](#backfilling-data). Then use:
+
+```ruby
+class ChangeSomeColumnNull < ActiveRecord::Migration[5.2]
def change
- add_reference :users, :reference, index: false
- add_index :users, :reference_id, algorithm: :concurrently
+ change_column_null :users, :some_column, false
end
end
```
-For polymorphic references, add a compound index on type and id.
+### Adding a json column
-### Adding a json column (Postgres)
+#### Bad
-There’s no equality operator for the `json` column type, which causes issues for `SELECT DISTINCT` queries.
+In Postgres, there’s no equality operator for the `json` column type, which causes issues for `SELECT DISTINCT` queries.
-If you’re on Postgres 9.4+, use `jsonb` instead.
+```ruby
+class AddPropertiesToUsers < ActiveRecord::Migration[5.2]
+ def change
+ add_column :users, :properties, :json
+ end
+end
+```
-If you must use `json`, replace all calls to `uniq` with a custom scope.
+#### Good
+Use `jsonb` instead.
+
```ruby
-class User < ApplicationRecord
- scope :uniq_on_id, -> { select("DISTINCT ON (users.id) users.*") }
+class AddPropertiesToUsers < ActiveRecord::Migration[5.2]
+ def change
+ add_column :users, :properties, :jsonb
+ end
end
```
-Then add the column:
+## Best Practices
+### Keeping non-unique indexes to three columns or less
+
+#### Bad
+
+Adding an index with more than three columns only helps on extremely large tables.
+
```ruby
-class AddJsonColumnToUsers < ActiveRecord::Migration[5.2]
+class AddSomeIndexToUsers < ActiveRecord::Migration[5.2]
def change
- safety_assured { add_column :users, :some_column, :json }
+ add_index :users, [:a, :b, :c, :d]
end
end
```
+#### Good
+
+```ruby
+class AddSomeIndexToUsers < ActiveRecord::Migration[5.2]
+ def change
+ add_index :users, [:a, :b, :c]
+ end
+end
+```
+
+> For Postgres, be sure to add them concurrently
+
## Assuring Safety
-To mark a step in the migration as safe, despite using method that might otherwise be dangerous, wrap it in a `safety_assured` block.
+To mark a step in the migration as safe, despite using a method that might otherwise be dangerous, wrap it in a `safety_assured` block.
```ruby
class MySafeMigration < ActiveRecord::Migration[5.2]
def change
safety_assured { remove_column :users, :some_column }
end
end
```
+Certain methods like `execute` and `change_table` cannot be inspected and are prevented from running by default. Make sure what you’re doing is really safe and use this pattern.
+
## Custom Checks
Add your own custom checks with:
```ruby
@@ -236,10 +466,12 @@
end
```
Use the `stop!` method to stop migrations.
+> Since `remove_column` always requires a `safety_assured` block, it’s not possible to add a custom check for `remove_column` operations
+
## Existing Migrations
To mark migrations as safe that were created before installing this gem, create an initializer with:
```ruby
@@ -310,10 +542,10 @@
- [Rails Migrations with No Downtime](https://pedro.herokuapp.com/past/2011/7/13/rails_migrations_with_no_downtime/)
- [Safe Operations For High Volume PostgreSQL](https://www.braintreepayments.com/blog/safe-operations-for-high-volume-postgresql/)
## Credits
-Thanks to Bob Remeika and David Waller for the [original code](https://github.com/foobarfighter/safe-migrations).
+Thanks to Bob Remeika and David Waller for the [original code](https://github.com/foobarfighter/safe-migrations) and [Sean Huber](https://github.com/LendingHome/zero_downtime_migrations) for the bad/good readme format.
## Contributing
Everyone is encouraged to help improve this project. Here are a few ways you can help: