README.md in pg_ha_migrations-1.6.0 vs README.md in pg_ha_migrations-1.7.0
- old
+ new
@@ -1,10 +1,10 @@
# PgHaMigrations
-[](https://travis-ci.org/braintree/pg_ha_migrations/)
+[](https://github.com/braintree/pg_ha_migrations/actions/workflows/ci.yml?query=branch%3Amaster+)
-We've documented our learned best practices for applying schema changes without downtime in the post [PostgreSQL at Scale: Database Schema Changes Without Downtime](https://medium.com/braintree-product-technology/postgresql-at-scale-database-schema-changes-without-downtime-20d3749ed680) on the [Braintree Product and Technology Blog](https://medium.com/braintree-product-technology). Many of the approaches we take and choices we've made are explained in much greater depth there than in this README.
+We've documented our learned best practices for applying schema changes without downtime in the post [PostgreSQL at Scale: Database Schema Changes Without Downtime](https://medium.com/paypal-tech/postgresql-at-scale-database-schema-changes-without-downtime-20d3749ed680) on the [PayPal Technology Blog](https://medium.com/paypal-tech). Many of the approaches we take and choices we've made are explained in much greater depth there than in this README.
Internally we apply those best practices to our Rails applications through this gem which updates ActiveRecord migrations to clearly delineate safe and unsafe DDL as well as provide safe alternatives where possible.
Some projects attempt to hide complexity by having code determine the intent and magically do the right series of operations. But we (and by extension this gem) take the approach that it's better to understand exactly what the database is doing so that (particularly long running) operations are not a surprise during your deploy cycle.
@@ -32,11 +32,11 @@
## Usage
### Rollback
-Because we require that ["Rollback strategies do not involve reverting the database schema to its previous version"](https://medium.com/braintree-product-technology/postgresql-at-scale-database-schema-changes-without-downtime-20d3749ed680#360a), PgHaMigrations does not support ActiveRecord's automatic migration rollback capability.
+Because we require that ["Rollback strategies do not involve reverting the database schema to its previous version"](https://medium.com/paypal-tech/postgresql-at-scale-database-schema-changes-without-downtime-20d3749ed680#360a), PgHaMigrations does not support ActiveRecord's automatic migration rollback capability.
Instead we write all of our migrations with only an `def up` method like:
```
def up
@@ -66,11 +66,11 @@
When `unsafe_*` migration methods support checks of this type you can bypass the checks by passing an `:allow_dependent_objects` key in the method's `options` hash containing an array of dependent object types you'd like to allow. Until 2.0 none of these checks will run by default, but you can opt-in by setting `config.check_for_dependent_objects = true` [in your configuration initializer](#configuration).
Similarly we believe the `force: true` option to ActiveRecord's `create_table` method is always unsafe, and therefore we disallow it even when calling `unsafe_create_table`. This option won't be enabled by default until 2.0, but you can opt-in by setting `config.allow_force_create_table = false` [in your configuration initializer](#configuration).
-[Running multiple DDL statements inside a transaction acquires exclusive locks on all of the modified objects](https://medium.com/braintree-product-technology/postgresql-at-scale-database-schema-changes-without-downtime-20d3749ed680#cc22). For that reason, this gem [disables DDL transactions](./lib/pg_ha_migrations.rb:8) by default. You can change this by resetting `ActiveRecord::Migration.disable_ddl_transaction` in your application.
+[Running multiple DDL statements inside a transaction acquires exclusive locks on all of the modified objects](https://medium.com/paypal-tech/postgresql-at-scale-database-schema-changes-without-downtime-20d3749ed680#cc22). For that reason, this gem [disables DDL transactions](./lib/pg_ha_migrations.rb:8) by default. You can change this by resetting `ActiveRecord::Migration.disable_ddl_transaction` in your application.
The following functionality is currently unsupported:
- Rollbacks
- Generators
@@ -218,10 +218,173 @@
```ruby
unsafe_remove_constraint :table, name: :constraint_table_on_column_like_example
```
+#### safe\_create\_partitioned\_table
+
+Safely create a new partitioned table using [declaritive partitioning](https://www.postgresql.org/docs/current/ddl-partitioning.html#DDL-PARTITIONING-DECLARATIVE).
+
+```ruby
+# list partitioned table using single column as partition key
+safe_create_partitioned_table :table, type: :list, partition_key: :example_column do |t|
+ t.text :example_column, null: false
+end
+
+# range partitioned table using multiple columns as partition key
+safe_create_partitioned_table :table, type: :range, partition_key: [:example_column_a, :example_column_b] do |t|
+ t.integer :example_column_a, null: false
+ t.integer :example_column_b, null: false
+end
+
+# hash partitioned table using expression as partition key
+safe_create_partitioned_table :table, :type: :hash, partition_key: ->{ "(example_column::date)" } do |t|
+ t.datetime :example_column, null: false
+end
+```
+
+The identifier column type is `bigserial` by default. This can be overridden, as you would in `safe_create_table`, by setting the `id` argument:
+
+```ruby
+safe_create_partitioned_table :table, id: :serial, type: :range, partition_key: :example_column do |t|
+ t.date :example_column, null: false
+end
+```
+
+In PostgreSQL 11+, primary key constraints are supported on partitioned tables given the partition key is included. On supported versions, the primary key is inferred by default (see [available options](#available-options)). This functionality can be overridden by setting the `infer_primary_key` argument.
+
+```ruby
+# primary key will be (id, example_column)
+safe_create_partitioned_table :table, type: :range, partition_key: :example_column do |t|
+ t.date :example_column, null: false
+end
+
+# primary key will not be created
+safe_create_partitioned_table :table, type: :range, partition_key: :example_column, infer_primary_key: false do |t|
+ t.date :example_column, null: false
+end
+```
+
+#### safe\_partman\_create\_parent
+
+Safely configure a partitioned table to be managed by [pg\_partman](https://github.com/pgpartman/pg_partman).
+
+This method calls the [create\_parent](https://github.com/pgpartman/pg_partman/blob/master/doc/pg_partman.md#creation-functions) partman function with some reasonable defaults and a subset of user-defined overrides.
+
+The first (and only) positional argument maps to `p_parent_table` in the `create_parent` function.
+
+The rest are keyword args with the following mappings:
+
+- `partition_key` -> `p_control`. Required: `true`
+- `interval` -> `p_interval`. Required: `true`
+- `template_table` -> `p_template_table`. Required: `false`. Partman will create a template table if not defined.
+- `premake` -> `p_premake`. Required: `false`. Partman defaults to `4`.
+- `start_partition` -> `p_start_partition`. Required: `false`. Partman defaults to the current timestamp.
+
+Note that we have chosen to require PostgreSQL 11+ and hardcode `p_type` to `native` for simplicity, as previous PostgreSQL versions are end-of-life.
+
+Additionally, this method allows you to configure a subset of attributes on the record stored in the [part\_config](https://github.com/pgpartman/pg_partman/blob/master/doc/pg_partman.md#tables) table.
+These options are delegated to the `unsafe_partman_update_config` method to update the record:
+
+- `infinite_time_partitions`. Partman defaults this to `false` but we default to `true`
+- `inherit_privileges`. Partman defaults this to `false` but we default to `true`
+- `retention`. Partman defaults this to `null`
+- `retention_keep_table`. Partman defaults this to `true`
+
+With only the required args:
+
+```ruby
+safe_create_partitioned_table :table, type: :range, partition_key: :created_at do |t|
+ t.timestamps null: false
+end
+
+safe_partman_create_parent :table, partition_key: :created_at, interval: "weekly"
+```
+
+With custom overrides:
+
+```ruby
+safe_create_partitioned_table :table, type: :range, partition_key: :created_at do |t|
+ t.timestamps null: false
+ t.text :some_column
+end
+
+# Partman will reference the template table to create unique indexes on child tables
+safe_create_table :table_template, id: false do |t|
+ t.text :some_column, index: {unique: true}
+end
+
+safe_partman_create_parent :table,
+ partition_key: :created_at,
+ interval: "weekly",
+ template_table: :table_template,
+ premake: 10,
+ start_partition: Time.current + 1.month,
+ infinite_time_partitions: false,
+ inherit_privileges: false
+```
+
+#### unsafe\_partman\_create\_parent
+
+We have chosen to flag the use of `retention` and `retention_keep_table` as an unsafe operation.
+While we recognize that these options are useful, we think they fit in the same category as `drop_table` and `rename_table`, and are therefore unsafe from an application perspective.
+If you wish to define these options, you must use this method.
+
+```ruby
+safe_create_partitioned_table :table, type: :range, partition_key: :created_at do |t|
+ t.timestamps null: false
+end
+
+unsafe_partman_create_parent :table,
+ partition_key: :created_at,
+ interval: "weekly",
+ retention: "60 days",
+ retention_keep_table: false
+```
+
+#### safe\_partman\_update\_config
+
+There are some partitioning options that cannot be set in the call to `create_parent` and are only available in the `part_config` table.
+As mentioned previously, you can specify these args in the call to `safe_partman_create_parent` or `unsafe_partman_create_parent` which will be delegated to this method.
+Calling this method directly will be useful if you need to modify your partitioned table after the fact.
+
+Allowed keyword args:
+
+- `infinite_time_partitions`
+- `inherit_privileges`
+- `premake`
+- `retention`
+- `retention_keep_table`
+
+Note that we detect if the value of `inherit_privileges` is changing and will automatically call `safe_partman_reapply_privileges` to ensure permissions are propagated to existing child partitions.
+
+```ruby
+safe_partman_update_config :table,
+ infinite_time_partitions: false,
+ inherit_privileges: false,
+ premake: 10
+```
+
+#### unsafe\_partman\_update\_config
+
+As with creating a partman parent table, we have chosen to flag the use of `retention` and `retention_keep_table` as an unsafe operation.
+If you wish to define these options, you must use this method.
+
+```ruby
+unsafe_partman_update_config :table,
+ retention: "60 days",
+ retention_keep_table: false
+```
+
+#### safe\_partman\_reapply\_privileges
+
+If your partitioned table is configured with `inherit_privileges` set to `true`, use this method after granting new roles / privileges on the parent table to ensure permissions are propagated to existing child partitions.
+
+```ruby
+safe_partman_reapply_privileges :table
+```
+
### Utilities
#### safely\_acquire\_lock\_for\_table
Safely acquire a lock for a table.
@@ -272,12 +435,13 @@
#### Available options
- `disable_default_migration_methods`: If true, the default implementations of DDL changes in `ActiveRecord::Migration` and the PostgreSQL adapter will be overridden by implementations that raise a `PgHaMigrations::UnsafeMigrationError`. Default: `true`
- `check_for_dependent_objects`: If true, some `unsafe_*` migration methods will raise a `PgHaMigrations::UnsafeMigrationError` if any dependent objects exist. Default: `false`
-- `prefer_single_step_column_addition_with_default`: If `true`, raise an error when adding a column and separately setting a constant default value for that column in the same migration. Default: `false`
-- 'allow_force_create_table`: If false, the `force: true` option to ActiveRecord's `create_table` method is disallowed. Default: `true`
+- `prefer_single_step_column_addition_with_default`: If true, raise an error when adding a column and separately setting a constant default value for that column in the same migration. Default: `false`
+- `allow_force_create_table`: If false, the `force: true` option to ActiveRecord's `create_table` method is disallowed. Default: `true`
+- `infer_primary_key_on_partitioned_tables`: If true, the primary key for partitioned tables will be inferred on PostgreSQL 11+ databases (identifier column + partition key columns). Default: `true`
### Rake Tasks
Use this to check for blocking transactions before migrating.
@@ -303,10 +467,10 @@
Running tests will automatically create a test database in the locally running Postgres server. You can find the connection parameters in `spec/spec_helper.rb`, but setting the environment variables `PGHOST`, `PGPORT`, `PGUSER`, and `PGPASSWORD` will override the defaults.
To install this gem onto your local machine, run `bundle exec rake install`.
-To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+To release a new version, update the version number in `version.rb`, commit the change, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
Note: if while releasing the gem you get the error ``Your rubygems.org credentials aren't set. Run `gem push` to set them.`` you can more simply run `gem signin`.
## Contributing