README.md in knife-topo-1.1.2 vs README.md in knife-topo-2.0.1
- old
+ new
@@ -14,205 +14,228 @@
like to manage its dynamic configuration (e.g. changing software versions)
through a single (json) configuration file. It may also be useful
if you are regularly bringing up multi-node systems with similar
topologies but differences in their configuration details.
+This plugin can be used in conjunction with the
+[topo cookbook](http://github.com/christinedraper/topo-cookbook)
+to configure dynamically deployed topologies of nodes (e.g. in AWS).
+In this approach, use knife-topo to set up the topology details.
+Use the 'topo' cookbook to let nodes pull their own detailed configuration
+from the topology data bag on the Chef server.
-# Installation #
+# Changes from V1 #
-[Download the latest knife-topo release](http://github.com/christinedraper/knife-topo/releases/latest),
-unzip and copy `lib/chef/knife` into your plugin directory, e.g.:
+## Attribute setting strategy
- $ unzip knife-topo-1.0.0.zip -d ~
- $ cd ~/knife-topo-1.0.0
- $ mkdir -p ~/.chef/plugins/knife
- $ cp lib/chef/knife/* ~/.chef/plugins/knife
+V2 introduces the notion of an attribute setting strategy. Instead of
+specifying node attributes and cookbook attributes separately, you
+specify one set of attributes and the method by which
+they should be set on the node (e.g. 'direct_to_node' or 'via_cookbook').
-or install knife-topo as a gem
+The purpose of this change is to make it easier to support new
+methods of setting attributes such as policy files, and also making it
+easier to switch between methods.
- $ gem install knife-topo
+## Node type
-You may need to use `chef gem install knife-topo` (to install into the
-embedded chefdk gem path), and you may need `sudo` depending on your setup.
+A node type can be specified for a node. The node type is used
+by the 'topo' cookbook to identify the right configuration to use. It
+is also used in the 'via cookbook' method to support attributes that
+vary by node type.
-This plugin has been tested with Chef Version 11.12 and 12.0.3 on Ubuntu 12.04 and 14.04 LTS,
-and run on Windows and Mac.
+## One topology per file
-Note: I've encountered a case (on a Mac) where knife was not configured to use
- gems on the gem path. If the gem install succeeds but `knife topo`
- is not a recognized knife command, try the first approach (copy
-the ruby plugin scripts into ~/.chef/plugins/knife) or install knife topo using the embedded gem:
+To reduce complexity, V2 no longer supports multiple topologies in a
+JSON file.
- $ /opt/chef/embedded/bin/gem install knife-topo
+## V1 -> V2 Migration
+V1 topology JSON files can be converted to V2 by importing them and
+then exporting them. knife-topo will auto-detect V1 format files on
+import. You can also explicitly specify the input format using
+`knife topo import sometopo.json --input-format 'topo_v1'`
+
+# Installation #
+
+Install knife-topo as a gem
+
+```
+ $ chef gem install knife-topo
+```
+
# Usage #
-Define one or more topologies in a [topology file](#topology-file). Import
+Define each topology in a [topology file](#topology-file). Import
that file into your Chef workspace using [knife topo import](#import),
-then create and bootstrap the nodes using a single command [knife topo create](#create).
-Update the topology file as the configuration changes (e.g., when you
-need to update software versions), import those changes and run one command
-[knife topo update](#update) to update all of the nodes.
+then create the topology [knife topo create](#create), specifying
+the '--bootstrap' option if you want to bootstrap all of the nodes.
+Update the topology file as the configuration changes, import those
+changes [knife topo import](#import) and run
+[knife topo update](#update) to update the topology.
+
# Getting Started #
Try out this plugin using a [test-repo](test-repo) provided in the knife-topo github repository.
[Download the latest knife-topo release](http://github.com/christinedraper/knife-topo/releases/latest)
and unzip it, then follow the [Instructions](test-repo/Instructions.md) for the example.
The instructions assume you have [chefDK](https://downloads.chef.io/chef-dk/)
- or equivalent installed and working with Vagrant and VirtualBox, but
- none of these are requirements to use the knife-topo plugin.
+ installed and working with Vagrant and VirtualBox.
If you're the sort of person who just wants to jump in and try it, here's some hints.
Generate a topology file for a topology called test1 from existing nodes node1 and node2:
- knife topo export test1 node1 node2 > topology.json
+ knife topo export node1 node2 --topo test1 > test1.json
Import a topology json file, generating all of the necessary artifacts in your workspace:
- knife topo import topology1.json
+ knife topo import test1.json
Create the topology using existing nodes:
knife topo create test1
-Create the topology bootstrapping new nodes in vagrant (you will need to add the
+Create the topology, bootstrapping new nodes in vagrant (you will need to add the
host details for bootstrap to the file before importing):
knife topo create test1 --bootstrap -xvagrant -Pvagrant --sudo
# Topology File <a name="topology-file"></a>#
-See the [example topology file](test-repo/topology.json)
+See the [example topology file](test-repo/test1.json)
-The topology file contains a single topology, or an array of topologies.
+The topology file contains a single topology.
Each topology has some overall properties, an array of nodes and
an array defining topology cookbook attributes.
## Overall Topology Properties <a name="topology-properties"></a>
```
{
"name": "test1",
+ "node_type": "appserver",
"chef_environment": "test",
- "tags": ["system_sys1", "phase_test" ],
+ "tags": [ "testsys" ],
"normal": {
- "owner": {
- "name": "Christine Draper"
- }
+ "owner": {
+ "name": "Christine Draper"
+ }
},
"nodes" : [
...
- ],
- "cookbook_attributes" : [
]
}
```
+
The `name` is how you will refer to the topology in the
`knife topo` subcommands.
The `chef-environment` and `normal` attributes defined
here will be applied to all nodes in the topology, unless alternative
values are provided for a specific node. The `tags`
will be added to each node.
+
+### Attribute setting strategy
+
+The default strategy for setting attributes is `direct_to_node`.
+In this strategy, normal attributes are set directly on the nodes when
+the topology is created, bootstrapped or updated. Attributes with
+other priorities are ignored.
+
+The `strategy` field can be set to 'via_cookbook', in which case
+additional `strategy_data` can be provided to specify a cookbok
+and attribute filename.
+```
+ {
+ "name": "test1",
+ ...
+ "strategy" : "via_cookbook",
+ "strategy_data": {
+ "cookbook": "topo_test1",
+ "filename": "softwareversion",
+ }
+ }
+```
+
+In this strategy, the cookbook and attribute file are generated
+in the local workspace when the topology is imported, and uploaded
+to the server when the topology is created or updated. Attributes
+can have any valid priority.
## Node List <a name="node-list"></a>
Each topology contains a list of `nodes`.
```
{
"name": "test1",
...
"nodes": [
{
- "name": "buildserver01",
- "ssh_host": "192.168.1.201",
- "ssh_port": "2224",
- "chef_environment": "dev",
- "run_list": ["role[base-ubuntu]", "ypo::db", "recipe[ypo::appserver]"],
- "normal": {
- "topo" : {
- node_type": "buildserver"
- }
- },
- "tags": ["build"]
+ "name": "buildserver01",
+ "node_type" : "buildserver",
+ "ssh_host": "192.168.1.201",
+ "ssh_port": "2224",
+ "chef_environment": "dev",
+ "run_list": [
+ "role[base-ubuntu]",
+ "ypo::db",
+ "recipe[ypo::appserver]"
+ ],
+ ... node attributes, see below ...,
+ "tags": [ "build" ]
},
...
]
}
```
Within `nodes`, the `name` field is the node name that will be used in Chef.
-The fields `chef_environment`, `run_list`, `tags` and the attributes
-within `normal` will also be applied to the node in Chef. All of these
+The fields `chef_environment`, `run_list` and `tags`
+will also be applied to the node in Chef. All of these
fields are optional.
-The `ssh_host` and `ssh_port` are optional fields that are used to
+The `node_type` sets the node attribute `normal['topo']['node_type']`.
+This attribute is used in the 'via_cookbook' strategy to specify
+attributes that apply to only nodes of that type.
+
+The `ssh_host` and `ssh_port` fields are used to
bootstrap a node.
-## Topology Cookbook Attributes <a name="cookbook-attributes"></a>
+## Node Attributes <a name="node-attributes"></a>
-Each topology may have attributes that are set via
-an attribute file in a topology-specific cookbook. Each
-attribute file is described in an entry in the 'cookbook_attributes'
-array.
+Each topology may have attributes that are set on each node
+according to the attribute setting strategy. The attribute names and
+values are specified by priority ('default', 'normal', 'override').
```
- "cookbook_attributes": [
+ "nodes": [
{
- "cookbook": "testsys_test1",
- "filename": "softwareversion",
+ "name": "buildserver01",
"normal":
{
"nodejs":
{
- "version": "0.28"
+ "version": "0.10.40"
},
"testapp":
{
"version": "0.0.3"
},
"mongodb":
{
- "package_version": "2.6.1"
+ "package_version": "2.6.9"
}
- },
- "conditional" : [
- {
- "qualifier": "node_type",
- "value" : "buildserver",
- "normal":
- {
- "mongodb":
- {
- "package_version": "2.5.1"
- }
- }
- }
- ]
+ }
}
]
```
-Attributes listed directly under an attribute priority (e.g. 'normal'
-in the above) will generate an entry in the attribute file such as:
-
- normal['mongodb'][package_version] = "2.6.1"
-
-Attributes listed under the `conditional` property will generate an
-entry in the attribute file such as:
-
-```
- if (node['topo']['node_type'] == "buildserver")
- normal['mongodb']['package_version'] = "2.5.1"
- end
-```
-
# Subcommands <a name="subcommands"></a>
The main subcommands for `knife topo` are:
* [knife topo import](#import) - Import one or more into your workspace/local repo
@@ -221,12 +244,10 @@
The additional subcommands can also be useful, depending on your
workflow:
* [knife topo bootstrap](#bootstrap)- Bootstraps a topology of nodes
-* [knife topo cookbook create](#cookbook-create) - Generate the topology cookbooks
-* [knife topo cookbook upload](#cookbook-upload) - Upload the topology cookbooks
* [knife export](#export) - Export data from a topology (or from nodes that you want in a topology)
* [knife topo list](#list) - List the topologies
* [knife topo search](#search) - Search for nodes that are in a topology, or in no topology
* [knife topo delete](#delete) - Delete a topology, but not the nodes in the topology
@@ -255,70 +276,29 @@
The knife topo bootstrap subcommand supports the following additional options.
Option | Description
------------ | -----------
+--overwrite | Re-bootstrap existing nodes
See [knife bootstrap](http://docs.opscode.com/knife_bootstrap.html) | Options supported by `knife bootstrap` are passed through to the bootstrap command
### Examples:
The following will bootstrap nodes in the test1 topology, using a
user name of vagrant, password of vagrant, and running using sudo.
$ knife topo bootstrap test1 -x vagrant -P vagrant --sudo
-
-## knife topo cookbook create <a name="cookbook-create"></a>
-
- knife topo cookbook create TOPOLOGY
-
-Generates the topology cookbook attribute files and attributes described in the
-[cookbook_attributes](#cookbook-attributes) property.
-
-### Options:
-
-The knife topo cookbook create subcommand supports the following additional options.
-
-Option | Description
------------- | -----------
-See [knife cookbook create](http://docs.opscode.com/chef/knife.html#cookbook) | Options supported by `knife cookbook create` are passed through
-
-### Examples:
-The following will generate the topology cookbook attribute files for
-topology test1.
-
- $ knife topo cookbook create test1
-
-## knife topo cookbook upload <a name="cookbook-upload"></a>
-
- knife topo cookbook upload TOPOLOGY
-
-Uploads the topology cookbook attribute files.
-
-### Options:
-
-The knife topo cookbook upload subcommand supports the following additional options.
-
-Option | Description
------------- | -----------
-See [knife cookbook upload](http://docs.opscode.com/chef/knife.html#cookbook) | Options supported by `knife cookbook upload` are passed through
-
-
-### Examples:
-The following will generate the topology cookbook attribute files for
-topology test1.
-
- $ knife topo cookbook create test1
-
## knife topo create <a name="create"></a>
knife topo create TOPOLOGY
Creates the specified topology in the chef server as an item in the
topology data bag. Creates the chef environment associated
-with the topology, if it does not already exist. Uploads any
-topology cookbooks. Updates existing nodes based on the topology
+with the topology, if it does not already exist. Uploads the
+topology cookbook, if using the 'via_cookbook' method.
+Updates existing nodes based on the topology
information. New nodes will be created if the bootstrap option is
specified.
### Options:
@@ -327,10 +307,11 @@
Option | Description
------------ | -----------
--bootstrap | Bootstrap the topology (see [topo bootstrap](#bootstrap))
See [knife bootstrap](http://docs.opscode.com/knife_bootstrap.html) | Options supported by `knife bootstrap` are passed through to the bootstrap command
--disable-upload | Do not upload topology cookbooks
+--overwrite | Re-bootstrap existing nodes
### Examples:
The following will create the 'test1' topology, and bootstrap it.
$ knife topo create test1 --bootstrap
@@ -348,11 +329,11 @@
remove them from the topology by removing the `['topo']['name']` attribute
which is used by `knife topo search`.
## knife topo export <a name="export"></a>
- knife topo export [ NODE ... ]
+ knife topo export NODE [ NODE ... ]
Exports the nodes into a topology JSON.
If the topology does not already exist,
an outline for a new topology will be exported. The exported JSON
@@ -368,43 +349,39 @@
The knife topo export subcommand supports the following additional options.
Option | Description
------------ | -----------
--topo | Name of the topology to export (defaults to 'topo1')
---all | Export all topologies
--min-priority | Only export attributes with a priority equal or above this priority.
### Examples:
The following will export the data for nodes n1 and n2 as part of a
topology called 'my_topo':
- $ knife topo export n1 n2 --topo my_topo > new_topo.json
+ $ knife topo export n1 n2 --topo my_topo > my_topo.json
-The following will export all existing topologies to a file called 'all_topos.json'.
-
- $ knife topo export --all > all_topos.json
-
The following will create an outline for a new topology called
'christine_test', or export the current details if it already exists:
$ knife topo export --topo christine_test > christine_test.json
## knife topo import <a name="import"></a>
- knife topo import [ TOPOLOGY_FILE [ TOPOLOGY ... ]]
+ knife topo import [ TOPOLOGY_FILE ]
-Imports the topologies from a
+Imports data bag items containing the topologies from a
[topology file](#topology-file) into the local repo. If no topology
file is specified, attempts to read from a file called 'topology.json'
-in the current directory. Generates the topology cookbook attribute
-files and attributes described in the 'cookbook_attributes' property.
+in the current directory. Generates additional artifacts (e.g.
+topology cookbook attribute file) where needed.
### Examples:
-The following will import all topologies defined in the 'topology.json' file.
+The following will import the topology or topologies defined in the
+'topology.json' file.
$ knife topo import topology.json
The following will import the 'test1' topology
defined in the 'topology.json' file.
@@ -482,10 +459,10 @@
# License #
Author:: Christine Draper (christine_draper@thirdwaveinsights.com)
-Copyright:: Copyright (c) 2014-2015 ThirdWave Insights, LLC
+Copyright:: Copyright (c) 2014-2016 ThirdWave Insights, LLC
License:: Apache License, Version 2.0
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.