README.md in strong_migrations-1.7.0 vs README.md in strong_migrations-1.8.0

- old
+ new

@@ -36,11 +36,11 @@ Active Record caches attributes, which causes problems when removing columns. Be sure to ignore the column: class User < ApplicationRecord - self.ignored_columns = ["name"] + self.ignored_columns += ["name"] end Deploy the code, then wrap this step in a safety_assured { ... } block. class RemoveColumn < ActiveRecord::Migration[7.1] @@ -58,29 +58,30 @@ ## Checks Potentially dangerous operations: - [removing a column](#removing-a-column) -- [adding a column with a default value](#adding-a-column-with-a-default-value) -- [backfilling data](#backfilling-data) -- [adding a stored generated column](#adding-a-stored-generated-column) - [changing the type of a column](#changing-the-type-of-a-column) - [renaming a column](#renaming-a-column) - [renaming a table](#renaming-a-table) - [creating a table with the force option](#creating-a-table-with-the-force-option) +- [adding an auto-incrementing column](#adding-an-auto-incrementing-column) [unreleased] +- [adding a stored generated column](#adding-a-stored-generated-column) - [adding a check constraint](#adding-a-check-constraint) - [executing SQL directly](#executing-SQL-directly) +- [backfilling data](#backfilling-data) Postgres-specific checks: - [adding an index non-concurrently](#adding-an-index-non-concurrently) - [adding a reference](#adding-a-reference) - [adding a foreign key](#adding-a-foreign-key) - [adding a unique constraint](#adding-a-unique-constraint) - [adding an exclusion constraint](#adding-an-exclusion-constraint) - [adding a json column](#adding-a-json-column) - [setting NOT NULL on an existing column](#setting-not-null-on-an-existing-column) +- [adding a column with a default value](#adding-a-column-with-a-default-value) Config-specific checks: - [changing the default value of a column](#changing-the-default-value-of-a-column) @@ -108,11 +109,11 @@ 1. Tell Active Record to ignore the column from its cache ```ruby class User < ApplicationRecord - self.ignored_columns = ["some_column"] + self.ignored_columns += ["some_column"] end ``` 2. Deploy the code 3. Write a migration to remove the column (wrap in `safety_assured` block) @@ -126,97 +127,10 @@ ``` 4. Deploy and run the migration 5. Remove the line added in step 1 -### Adding a column with a default value - -#### Bad - -In earlier versions of Postgres, MySQL, and MariaDB, adding a column with a default value to an existing table causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB. - -```ruby -class AddSomeColumnToUsers < ActiveRecord::Migration[7.1] - def change - add_column :users, :some_column, :text, default: "default_value" - end -end -``` - -In Postgres 11+, MySQL 8.0.12+, and MariaDB 10.3.2+, this no longer requires a table rewrite and is safe (except for volatile functions like `gen_random_uuid()`). - -#### Good - -Instead, add the column without a default value, then change the default. - -```ruby -class AddSomeColumnToUsers < ActiveRecord::Migration[7.1] - def up - add_column :users, :some_column, :text - change_column_default :users, :some_column, "default_value" - end - - def down - remove_column :users, :some_column - end -end -``` - -See the next section for how to backfill. - -### Backfilling data - -#### Bad - -Active Record creates a transaction around each migration, and backfilling in the same transaction that alters a table keeps the table locked for the [duration of the backfill](https://wework.github.io/data/2015/11/05/add-columns-with-default-values-to-large-tables-in-rails-postgres/). - -```ruby -class AddSomeColumnToUsers < ActiveRecord::Migration[7.1] - def change - add_column :users, :some_column, :text - User.update_all some_column: "default_value" - end -end -``` - -Also, running a single query to update data can cause issues for large tables. - -#### Good - -There are three keys to backfilling safely: batching, throttling, and running it outside a transaction. Use the Rails console or a separate migration with `disable_ddl_transaction!`. - -```ruby -class BackfillSomeColumn < ActiveRecord::Migration[7.1] - disable_ddl_transaction! - - def up - User.unscoped.in_batches do |relation| - relation.update_all some_column: "default_value" - sleep(0.01) # throttle - end - end -end -``` - -### Adding a stored generated column - -#### Bad - -Adding a stored generated column causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB. - -```ruby -class AddSomeColumnToUsers < ActiveRecord::Migration[7.1] - def change - add_column :users, :some_column, :virtual, type: :string, as: "...", stored: true - end -end -``` - -#### Good - -Add a non-generated column and use callbacks or triggers instead (or a virtual generated column with MySQL and MariaDB). - ### Changing the type of a column #### Bad Changing the type of a column causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB. @@ -341,10 +255,48 @@ end ``` If you intend to drop an existing table, run `drop_table` first. +### Adding an auto-incrementing column + +#### Bad + +Adding an auto-incrementing column (`serial`/`bigserial` in Postgres and `AUTO_INCREMENT` in MySQL and MariaDB) causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB. + +```ruby +class AddIdToCitiesUsers < ActiveRecord::Migration[7.1] + def change + add_column :cities_users, :id, :primary_key + end +end +``` + +With MySQL and MariaDB, this can also [generate different values on replicas](https://dev.mysql.com/doc/mysql-replication-excerpt/8.0/en/replication-features-auto-increment.html) if using statement-based replication. + +#### Good + +Create a new table and migrate the data with the same steps as [renaming a table](#renaming-a-table). + +### Adding a stored generated column + +#### Bad + +Adding a stored generated column causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres, and writes are blocked in MySQL and MariaDB. + +```ruby +class AddSomeColumnToUsers < ActiveRecord::Migration[7.1] + def change + add_column :users, :some_column, :virtual, type: :string, as: "...", stored: true + end +end +``` + +#### Good + +Add a non-generated column and use callbacks or triggers instead (or a virtual generated column with MySQL and MariaDB). + ### Adding a check constraint :turtle: Safe by default available #### Bad @@ -395,10 +347,44 @@ safety_assured { execute "..." } end end ``` +### Backfilling data + +#### Bad + +Active Record creates a transaction around each migration, and backfilling in the same transaction that alters a table keeps the table locked for the [duration of the backfill](https://wework.github.io/data/2015/11/05/add-columns-with-default-values-to-large-tables-in-rails-postgres/). + +```ruby +class AddSomeColumnToUsers < ActiveRecord::Migration[7.1] + def change + add_column :users, :some_column, :text + User.update_all some_column: "default_value" + end +end +``` + +Also, running a single query to update data can cause issues for large tables. + +#### Good + +There are three keys to backfilling safely: batching, throttling, and running it outside a transaction. Use the Rails console or a separate migration with `disable_ddl_transaction!`. + +```ruby +class BackfillSomeColumn < ActiveRecord::Migration[7.1] + disable_ddl_transaction! + + def up + User.unscoped.in_batches do |relation| + relation.update_all some_column: "default_value" + sleep(0.01) # throttle + end + end +end +``` + ### Adding an index non-concurrently :turtle: Safe by default available #### Bad @@ -663,9 +649,44 @@ execute 'ALTER TABLE "users" DROP CONSTRAINT "users_some_column_null"' end end end ``` + +### Adding a column with a default value + +#### Bad + +In earlier versions of Postgres, adding a column with a default value to an existing table causes the entire table to be rewritten. During this time, reads and writes are blocked in Postgres. + +```ruby +class AddSomeColumnToUsers < ActiveRecord::Migration[7.1] + def change + add_column :users, :some_column, :text, default: "default_value" + end +end +``` + +In Postgres 11+, this no longer requires a table rewrite and is safe (except for volatile functions like `gen_random_uuid()`). + +#### Good + +Instead, add the column without a default value, then change the default. + +```ruby +class AddSomeColumnToUsers < ActiveRecord::Migration[7.1] + def up + add_column :users, :some_column, :text + change_column_default :users, :some_column, "default_value" + end + + def down + remove_column :users, :some_column + end +end +``` + +Then [backfill the data](#backfilling-data). ### Changing the default value of a column #### Bad