README.md in cfn-flow-0.7.0 vs README.md in cfn-flow-0.8.0

- old
+ new

@@ -1,20 +1,24 @@ # cfn-flow -An opinionated command-line workflow for developing [AWS CloudFormation](https://aws.amazon.com/cloudformation/) templates and deploying stacks. +`cfn-flow` is an command-line tool for developing [AWS CloudFormation](https://aws.amazon.com/cloudformation/) templates and deploying stacks. -Track template changes in git and publish versioned releases to AWS S3. +It provides a *simple*, *standard*, and *flexible* process for using CloudFormation, ideal for DevOps-style organizations. -Deploy stacks using a standard, reliable process with extensible -configuration in git. - #### Opinions -1. *Optimize for onboarding.* The workflow should be simple to learn & understand. -2. *Optimize for happiness.* The workflow should be easy and enjoyable to use. -3. *Auditable changes.* Know who changed what when. Leverage git change history. -4. *Immutable releases.* The code in a release never changes. +`cfn-flow` introduces a consist, convenient workflow that encourages good template organization +and deploy practices. +1. *Optimize for happiness.* The workflow should be easy and enjoyable to use. +2. *Optimize for onboarding.* The workflow should be simple to learn & understand. +3. *Auditable changes.* Know who changed what when. Leverage git history. +4. *Immutable releases.* The code in a release never changes. To make a change, + launch a new stack. + +The features & implementation of `cfn-flow` itself must also be simple. This follows the Unix philosophy of "[worse is +better](http://www.jwz.org/doc/worse-is-better.html)". `cfn-flow` values a simple design and implementation, and being composable with other workflows over handling every edge case out of the box. + ## Installation Via [rubygems](https://rubygems.org/gems/cfn-flow): ``` gem install cfn-flow @@ -22,24 +26,87 @@ The `git` command is also needed. ## Usage -Poke around: ``` +# Get help cfn-flow help cfn-flow help COMMAND # E.g.: cfn-flow help deploy ``` -Launching a CloudFormation stack: +Launch a CloudFormation stack: ``` cfn-flow deploy production ``` +## How it works + +`cfn-flow` works from a directory containing a `cfn-flow.yml` config file, and a CloudFormation template. +Presumably your app code is in the same directory, but it doesn't have to be. + +There are two key concepts for `cfn-flow`: **services** and **environments**. + +#### Services + +A service is a name for your project and comprises a set of resources that +change together. Each service has it's own `cfn-flow.yml` config file. A service +can be instantiated as several distinct environments. + +For example, a `WebApp` service could have a CloudFormation template that +creates an ELB, LaunchConfig, and AutoScalingGroup resources. + +All the resources in a service change together. Deploying the `WebApp` +service to an environment will create a new ELB, LaunchConfig, and AutoScalingGroup. + +Resources that *do not* change across deploys are not part of the service (from +`cfn-flow`'s perspective). +Say all `WebApp` EC2 servers connect to a long-running RDS database. That +database is not part of the cfn-flow service because it should re-used across +deploys. The database is a *backing resource* the service uses; not part +of the service itself. + +#### Environments + +An environment is an particular instantiation of a service. For example, you +could deploy your `WebApp` service to both a `development` and `production` environment. + +`cfn-flow` is designed to support arbitrary environments like git supports +arbitrary branches. + +Then `CFN_FLOW_ENVIRONMENT` environment variable can be used in +`cfn-flow.yml` to use the environment in your template parameters. + +#### Deploying + +Deployments consist of launching a *new stack* in a particular environment, then +shutting down the old stack. For example: + +``` +cfn-flow deploy ENVIRONMENT --cleanup +``` + +This follows the [red/black](http://techblog.netflix.com/2013/08/deploying-netflix-api.html) +or [blue/green](http://martinfowler.com/bliki/BlueGreenDeployment.html) +deployment pattern. + +After verifying the new stack is working correctly, the deployer is expected to +delete the old stack. + +To roll back a bad deploy, simply delete the *new* stack, while the *old* +stack is running. + +Although CloudFormation supports updating existing stacks, `cfn-flow` prefers +launching immutable stacks. Stack updates are more difficult to test than new stacks; and there's less chance of a deployment error disrupting or breaking important resources. + +#### AWS credentials + +Set your AWS credentials so they can be found by the AWS SDK for Ruby ([details here](http://docs.aws.amazon.com/AWSSdkDocsRuby/latest/DeveloperGuide/set-up-creds.html)), e.g. using the `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment variables. + ## Configuration `cfn-flow` looks for `./cfn-flow.yml` for stack and template configuration. You can override this path by setting the `CFN_FLOW_CONFIG_PATH` environment variable to another path. @@ -53,10 +120,11 @@ # Minimal configuration for launching the stack. stack: # Stack name uses embedded ruby to support dynamic values stack_name: MyService-<%= Time.now.to_i %> # Required: *either* template_url or template_body + # NB: template_body is a local path to the template template_body: path/to/template.json # Alternatively: # template_url: https://MyS3Bucket.s3.amazonaws.com/MyPrefix/release/abc123/template.json ``` @@ -79,12 +147,20 @@ # Published url: https://MyS3Bucket.s3.amazonaws.com/My/S3/Prefix/<git sha>/my-cfn-template.json templates: bucket: MyS3Bucket s3_prefix: 'My/S3/Prefix' +## +# Stacks +# +# These are the arguments passed when launching a new stack. +# It's nearly identical to the create_stack args in the ruby sdk, except +# parameters and tags are hashes. See http://amzn.to/1M0nBuq + stack: stack_name: MyService-<%= Time.now.to_i %> + # NB: template_body is a local path to the template template_body: path/to/template.yml template_url: http://... parameters: # Your parameters, e.g.: vpcid: vpc-1234 @@ -95,130 +171,156 @@ capabilities: ["CAPABILITY_IAM"], # This stack does IAM stuff on_failure: "DO_NOTHING", # either DO_NOTHING, ROLLBACK, DELETE stack_policy_body: "StackPolicyBody", stack_policy_url: "StackPolicyURL", tags: + # Whatever you want. + # Note that `cfn-flow` automatically adds two tags: `CfnFlowService` and `CfnFlowEnvironment` TagKey: TagValue # Who launched this stack Deployer: <%= ENV['USER'] %> # Tag production and development environments for accounting BillingType: <%= ENV['CFN_FLOW_ENVIRONMENT'] == 'production' ? 'production' : 'development' %> ``` -#### Dev mode (default) +### UX improvements: -Dev mode allows you to quickly test template changes. -`cfn-flow` validates all templates and uploads them to your personal prefix, overwriting existing templates. +`cfn-flow` includes a few developer-friendly features: -Dev mode does not verify that your local changes are -committed to git (as opposed to release mode). +#### YAML > JSON -You should use dev mode for testing & verifying changes in non-production stacks. +`cfn-flow` lets you write templates in either JSON or +[YAML](http://www.yaml.org). YAML is a superset of JSON that allows a terser, +less cluttered syntax, inline comments, and code re-use with anchors (like +variables). YAML templates are transparently converted to JSON when uploaded to +S3 for use in CloudFormation stacks. +Note that you can use JSON snippets inside YAML templates. JSON is always valid +YAML. + +#### Embedded ruby in `cfn-flow.yml` + +To allow dynamic/programatic attributes, use +[ERB](https://en.wikipedia.org/wiki/ERuby) in `cfn-flow.yml`. For example: + +```yaml +stack: + name: my-stack-<%= Time.now.to_i %> + ... + parameters: + git_sha: <%= `git rev-parse --verify HEAD`.chomp %> ``` -# Set a personal name to prefix your templates. -export CFN_FLOW_DEV_NAME=aaron -# Validate and upload all CloudFormation templates in your working directory to -s3://my-bucket/dev/aaron/* -# NB that this overwrites existing templates in your CFN_FLOW_DEV_NAME -namespace. +## Usage -cfn-flow +### Working with stacks + +`cfn-flow` automatically sets two tags on any stack it launches: + +Name | Example value +--- | --- +CfnFlowService | `myapp` +CfnFlowEnvironment | `production` + +These tags let `cfn-flow` associate stacks back to services & environments. + +#### `cfn-flow deploy ENVIRONMENT` + +Launches a stack in ENVIRONMENT. E.g. `cfn-flow deploy production` + +Add the `--cleanup` option to be prompted to shut down other stacks in the environment. + +#### `cfn-flow list ENVIRONMENT` + +Show running stacks for ENVIRONMENT. + ``` +$ cfn-flow list production -You can launch or update test stacks using your dev template path to quickly test your -template changes. +myapp-production-aaa (CREATE_COMPLETE) +myapp-production-bbb (CREATE_FAILED) +``` -#### Release mode +#### `cfn-flow delete STACK` -Release mode publishes your templates to a versioned S3 path, and pushes a git -tag of the version. +Deletes a stack. ``` -# uploads templates to `s3://my-bucket/release/<git sha>/*` -tag -cfn-flow --release +$ cfn-flow delete myapp-production-aaa ``` -Release mode ensures there are no uncommitted changes in your git working -directory. +#### `cfn-flow show STACK` -Inspecting the differences between releases is possible using `git log` and `git -diff`. +Show the status of STACK. -## Configuration +#### `cfn-flow events STACK` -You can configure cfn-flow defaults by creating a `cfn-flow.yml` file in same -directory you run `cfn-flow` (presumably the root of your project). +List events for STACK -Settings in the configuration file are overridden by environment variables. And -environment variables are overridden by command line arguments. +Use the `--tail` option to poll for new events until the stack status is no +longer `*_IN_PROGRESS` +### Common workflows + +#### Deploying to production + ``` -# cfn-flow.yml in the root of your project -# You can specify an alternative path by setting the CFN_FLOW_CONFIG environment -# variable. -# -# All options in this config can be overridden with command line arguments ---- -# S3 bucket where templates are uploaded. No default. -# Override with CFN_FLOW_BUCKET env var -bucket: 'my-s3-bucket' +# Launch a new stack for the current git commit +$ cfn-flow deploy production +Launching stack myapp-production-abc123 +# ... wait for it to be ready -# S3 path prefix. Default: none -# Override with CFN_FLOW_TO env var -to: my/project/prefix +# See the other stacks +$ cfn-deploy list production -# Local path in which to recursively search for templates. Default: . -# Override with CFN_FLOW_FROM env var -from: my/local/prefix +myapp-production-abc123 CREATE_COMPLETE +myapp-production-xyz987 CREATE_COMPLETE -# AWS Region -# Override with AWS_REGION env var -region: us-east-1 # AWS region +# Shut down the old stack +$ cfn-flow delete myapp-production-xyz987 ``` -#### AWS credentials +### Launching a development environment -AWS credentials can only be set using the -`AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment variables; or by -using an EC2 instance's IAM role. +Launch a new stack for `myenv` environment +``` +cfn-flow deploy myenv +``` -## Sweet Features +### Working with templates -#### YAML > JSON +#### `cfn-flow validate` -`cfn-flow` lets you write templates in either JSON or -[YAML](http://www.yaml.org). YAML is a superset of JSON that allows a terser, -less cluttered syntax, inline comments, and code re-use with variables. YAML -templates are transparently converted to JSON when uploaded to S3 for use in -CloudFormation stacks. +``` +# Runs validate-template on all templates. +# returns an error on any failure. +# does not persist to S3 -#### Use versions in nested stack template URLs +$ cfn-flow validate path/to/template.yml +``` -`cfn-flow` works great with [nested stack -resources](http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-stack.html). Use [Fn::Join](http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-join.html) to construct the `TemplateURL` from a parameter: +#### `cfn-flow publish` +Publish templates to S3 with immutable release names, or overwrite "dev names" +for quicker testing. *This is only needed if you want to use nested stack resources.* + ``` -{ - "Type" : "AWS::CloudFormation::Stack", - "Properties" : { - "TemplateURL" : { - "Fn::Join" : [ ":", - [ "https://s3.amazonaws.com/my-bucket", {"Ref": "prefix"}, "my-template.json" ] - ] - } - } -} -``` +$ cfn-flow publish path/to/template.yml +# validates & uploads templates to dev path +# Env var CFN_FLOW_DEV_NAME=aaron +# E.g. https://mybucket.s3.amazonaws.com/myprefix/dev/aaron/mytemplate.yml -While testing, set the `prefix` parameter to a dev prefix like `dev/aaron`. When you're confident your changes work, release them with cfn-flow and change the `prefix` parameter to `release/<git sha>` for production. +$ cfn-flow upload --release +# validates & uploads templates for current git sha +# E.g. https://mybucket.s3.amazonaws.com/myprefix/deadbeef/mytemplate.yml -#### Continuous integration +$ cfn-flow upload --release=v1.0.0 +# Upload templates for an arbitrary release name +# E.g. https://mybucket.s3.amazonaws.com/myprefix/v1.0.0/mytemplate.yml +``` -#### Github commit status +## License -#### Minimal AWS credentials +Copyright Kickstarter, Inc. -TODO: example IAM policy +Released under an [MIT License](http://opensource.org/licenses/MIT).