README.md in bjn_inventory-1.3.1 vs README.md in bjn_inventory-1.5.1
- old
+ new
@@ -194,9 +194,174 @@
```ruby
system_type always 'ec2_instance'
```
+## Commands
+
+### Formatters
+
+The inventory can be output in different formats. Two formatters
+are provided which can display the inventory as model-conformant
+devices: either as a JSON array or an object keyed by an identifying
+field (`inventory-model`); or in the
+[Ansible Dynamic Inventory](http://docs.ansible.com/ansible/latest/intro_dynamic_inventory.html)
+format.
+
+The `refresh_inventory_data` formatter formats the inventory into
+groups and devices in a file tree. A `devices.json` index is produced,
+with all devices, and each device also gets a file in `devices/<key>.json`.
+In addition the groups are listed in a `groups.json` index, and each
+has a list of devices in `groups/<group>.json`. This facilitates sharing
+the inventory over the web, as well (though **bjn_inventory** is not
+itself a network service).
+
+### Downloaders
+
+The overall design of the software encourages you to download entries
+from inventory sources in a "close to raw" manner, and let
+the source merging and mapping rules transform the entries into
+devices. Because the core inventory generation offers no mapping,
+the downloader is also the point at which to filter out entries
+that should not be devices.
+
+For convenience, AWS downloaders are provided for instances
+(`aws-ec2-source`), classic ELB (`aws-elb-source`) and RDS
+(`aws-rds-source`) resources. Run each with valid AWS credentials in
+your `~/.aws/credentials` file to see what the output looks like.
+They each accept a `--filter` argument: in the EC2 downloader's case,
+this is passed to the AWS API to filter instances according to the
+attributes and tags it offers; in the case of the ELB downloader,
+the syntax is the same, but it is enforced by an internal rules-
+matching library. The RDS downloader also does filtering based only
+on tags, using the `--tag-filters` option.
+
+### Service Maps
+
+A common use case for device inventories is to be able to present
+service endpoints calculated dynamically from inventory. These could
+be used as-is, imported into a service discovery system, etc. To
+facilitate this, a special formatter is provided which maps devices
+and groups into "service endpoints", which are defined by a service
+map.
+
+A service map is a JSON object stored in a file (and provided to the
+`service-map` command via the `--map` option), where the keys are
+service prefixes and the values are objects. It can be arbitrarily
+deeply nested, with the deepest object in the tree being a service
+specifier.
+
+A service specifier consists of the special field `hosts`, the value
+of which is a list of groups. Each device in all the groups is added
+to the service under the preceding prefix. The other fields in the
+service specifier are arbitrary; each is passed through unchanged, so
+that the "leaf" of the service map consists of an object where the
+service specifier's key is joined with each of the specifier fields by
+a dot (`.`); and the `hosts` field has each device's endpoint listed
+with the `join_with` character (by default, a comma; or as a JSON
+array).
+
+The groups are determined by a group specification: similar to the
+`ansible-from` command, a JSON object (in the file given by the
+`--groups` option) with a `"group_by"` key, the value of which is a
+list of fields to create groups by. Note that if you use
+the `ansible-from` command, you can use the same groups file; but
+Ansible's `"groups"` key, which specifies groups of groups, is ignored.
+
+In the service prefix (the nested keys and objects which precede the
+"leaf" of the service map), device fields can be given with a dollar sign
+(`$`) prepended; these will be interpolated with the actual devices'
+field values. This also means that trees can be copied into the map multiple
+times, once for every unique value of the field in the relevant groups.
+
+The above description sounds complicated, but the result is a fairly
+simple way to map service endpoints into an arbitrarily complex tree, and
+a couple of examples are probably best to illustrate the point. Let's say
+you have three devices in your inventory: two in the `us-west-2` region
+and one in the `eu-west-1` region. In your `us-west-2` region, one instance
+has the `web` role and one has the `db` role. In the `us-west-1` region,
+you just have a webserver. Your whole inventory looks like this (for
+example, if you run `inventory-model --manifest manifest.json`:
+
+```javascript
+[
+ { "name": "web-01",
+ "roles": ["web"],
+ "region": "us-west-2" },
+ { "name": "db-01",
+ "roles": ["db"],
+ "region": "us-west-2" },
+ { "name": "web-02",
+ "roles": ["web"],
+ "region": "eu-west-1" }
+]
+```
+
+You create the following service map:
+
+
+```javascript
+{
+ "services": {
+ "$region": {
+ "www": {
+ "hosts": ["web"],
+ "port": 80
+ }
+ }
+ },
+ "monitor": {
+ "$region": {
+ "nagios": {
+ "hosts": ["web", "db"],
+ "key": "monitoring-key.rsa"
+ }
+ }
+ }
+}
+```
+
+When you run `service-map --map map.json --manifest manifest.json --hosts-field name`, you
+get the following output:
+
+```javascript
+{
+ "services": {
+ "us-west-2": {
+ "www.hosts": "10.0.1.1",
+ "www.port": 80
+ },
+ "eu-west-1": {
+ "www.hosts": "10.0.10.1",
+ "www.port": 80
+ }
+ },
+ "monitor": {
+ "us-west-2": {
+ "nagios.hosts": "10.0.1.1,10.0.1.2",
+ "nagios.key": "monitoring-key.rsa"
+ },
+ "eu-west-1": {
+ "nagios.hosts": "10.0.10.1",
+ "nagios.key": "monitoring-key.rsa"
+ }
+ }
+}
+```
+
+If you need to have a key in your service map that starts with a
+dollar sign, use two dollar signs instead.
+
+## TODO
+
+* The `refresh_inventory_data` formatter needs to changed to be
+ more parallel with the other formatters (`--ansible` should
+ be `--groups`, it should probably be named `inventory-files`
+ or something.
+* The `aws-rds-source` should be refactored to take the `--filters`
+ argument and use **BjnInventory::Util::Filter::JsonAws** like
+ `aws-elb-source`.
+
## Design Decisions
* No calculated fields in the model.
* No filtering: you can either filter the sources, or you can filter
the inventory after producing it