* Ansible Powerplay

  #+ATTR_HTML: title="Join the chat at https://gitter.im/flajann2/ansible-powerplay"
  [[https://gitter.im/flajann2/ansible-powerplay?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge][file:https://badges.gitter.im/flajann2/ansible-powerplay.svg]]

  Powerplay allows you to run multiple Ansible
  playbooks in parallel. Depending on how you organize
  your playbooks, this can be a solid win. I basically
  before this had been doing a playbook with multiple
  includes for other playbooks representing different
  servers in our stack. Playbook launching of playbooks
  is slow and very serial.

  Basically, the playbooks are all contained, so no
  interdependencies. And in my case, running in the
  cloud, so no reason why they can't be running in
  parallel

  Powerplay allows you to specify vars common to all
  playbooks, and also vars specific to some playbooks
  so by which you can make your setup very DRY.

  All the Ansible playbooks are executed in seperate
  processes, and thus avoiding a number of the "side
  effects" you would normally encounter with running
  multiple playbooks with Ansible includes.

  For example, here is Powerplay integrated with tmux:
  #+CAPTION: Powerplay writing to tmux panes, one pane per playbook.
  #+NAME: Powerplay Example
  [[./examples/powerplay_screenshot.jpeg]]

** Release Notes
   Please see [[RELEASE_NOTES.org][Release Notes]].
*** Hilights
**** DSL
     The version 1.x releases adds new features to
     the DSL, most notably, nestable groups, and being
     able to label each group as :sync or :async.
**** STDOUT from ansible-playbook
     The capture of the output from ansible-powerplay is 
     handled a bit more intelligently. If you do not specify the
     --tmux (or -m) option, all output is now currently captured
     by Powerplay and redumped to the console.

     Because you may still want to see the color from
     ansible-powerplay, you can alter ansible.cfg and 
     add the following line:

     + force_color = 1
     
     Please see [[New STDOUT capturing with 1.0.x][New STDOUT capturing with 1.0.x]]

**** Group Sequence Numers
     We now allow groups to have sequence numbers, as
     of 1.1. Basically, if you specify a sequence,
     the variable you designate will be assigned 
     a value in each of the sequence, with the
     group re-excuted. for example:

     #+begin_src ruby
     group :first, "async group with sequencing",
                 seq: { iter: [1, 5, 9, :dodo] } do
       book :nat, "nat.yml"
       book :dat, "dat.yml"
       book :rat, "rat.yml"
     end     
     #+end_src

     as you can see (in the development.play sample)
     the variable "iter" will be successively assigned
     the element in the [] array, with the underlying
     playbooks called. The is the function equivalent
     of the following:

     #+begin_src ruby
     group :first, "async group without sequencing" do
       configuration do
         iter 1
       end
       book :nat, "nat.yml"
       book :dat, "dat.yml"
       book :rat, "rat.yml"
     end

     group :first, "async group without sequencing" do
       configuration do
         iter 5
       end
       book :nat, "nat.yml"
       book :dat, "dat.yml"
       book :rat, "rat.yml"
     end     

     group :first, "async group without sequencing" do
       configuration do
         iter 9
       end
       book :nat, "nat.yml"
       book :dat, "dat.yml"
       book :rat, "rat.yml"
     end

     group :first, "async group without sequencing" do
       configuration do
         iter :dodo
       end
       book :nat, "nat.yml"
       book :dat, "dat.yml"
       book :rat, "rat.yml"
     end     
     #+end_src

     As you can see, the new sequencing can be quite 
     succinct.

** Features and Cavets
*** Integration with TMUX
    When running multiple Ansible Playbooks
    concurrently, one would like to be able to see the
    output of each in a reasonable manner. To faciliate
    this in this initial realse, we shall make heavy
    use of TMUX panes to dump the output.

    So basically, you need as many panes as you have
    concurrent Ansible Playbooks in this initial
    release. In subsequent releases, Curses will be
    directly leveraged to create "tabs" for the
    multiple output streams. We may even do this,
    still, through TMUX.

    Your input on this is strongly encouarged. We will
    not be supporting Screen at all. Sorry.

*** New STDOUT capturing with 1.0.x
    The new capture, while properly capturing the STDOUT
    of concurrent async runs, does not display until
    ansible-playbook completes. If you are like me, you'll
    like to see the progress as it runs. You still can using
    tmux with the --tmux (-m) option.

** DSL Terminology & Documentation
   Note that this is the DSL for version 1.x of
   PowerPlay. For 0.x, please see those tags in
   GitHub.

*** DSL
    The DSL is straightforward as possible,
    simple and elegant to allow you to write 
    your Powerplays in a DRY manner.

    For examples, please see the following:
    | [[examples/stack.play][stack.play]]       | This is loaded by default, and you must be in your current directory     |
    | [[examples/development.play][development.play]] | This is a fullblown Power Playbook for a hypothetical development stack. |
    | [[examples/production.play][production.play]]  | This is a fullblown Power Playbook for a hypothetical production stack.  |
    | [[examples/playbooks][playbooks]]        | Sample Ansible playbooks called by Powerplay.                            |

    To run the powerplay example:

    1. Install Ansible Powerplay
       + gem install ansible-powerplay
    2. Clone this project locally, then cd into the examples directory
       + git clone https://github.com/flajann2/ansible-powerplay.git
       + cd ansible-powerplay/examples
    3. source ansible-paths and run Powerplay
       + source ansible-paths.sh
       + powerplay play -p development -v2

    Note that I deliberately left a missing "elasticsearch.yml" so you
    can see how Powerplay handles the errors.
      
**** configuration
     You can intersperse configuration blocks
     anywhere, and the expected nested scoping
     will take effect.
**** playbooks
     playbooks are a collection of groups, and a group
     defaults to async mode for its members.

     Group are normally executed serially. This will
     allow you to organize your plays in an intelligent
     manner to deploy and manage resources and assets
     that may have to be done in a serial manner.
**** group
     A group is a collection of books or other groups
     that all execute in parallel by default.
     Books are required to be independent of 
     each other. If they are not, you can set
     them up to execute serially.

**** book
     A book has a direct correspondence to an Ansible
     playbook, and will execute that Yaml file
     given the configuration variables as parameters.

     Here is where var inheritance becomes useful.
     Note that all the configuration variables
     set at the time the book is called are all
     passed in as --extra-vars to Ansible Playbook.
     The Playbook may not need all the vars passed
     in, but care must be taken that no vars
     are used in a different manner than expected.
     We currently have no way of knowing which
     vars are needed or not, and to specifiy that 
     would make the syntax messy and loose some
     of the advantages of var inheritance.

** Installation
   Easy installation. From command-line:
   #+BEGIN_SRC bash
   gem install ansible-powerplay
   #+END_SRC

   Or from a gemfile:
   #+BEGIN_SRC ruby
   gem 'ansible-powerplay'
   #+END_SRC

** Use
   Basically, cd to the root of your Ansible directory,
   and a .play file (see the example at: [[https://github.com/flajann2/ansible-powerplay/blob/master/examples/stack.play][stack.play]].)

   You can place a config clause either globally,
   inside of playbooks, inside of groups, and the
   variable set up this way are inherited to the
   inner clauses, thus allowing you to keep your
   specifications DRYer.

   For example:
   #+BEGIN_SRC ruby
   # This is a global system configuration
   configuration :system do
     playbook_directory "playbooks"
   end
   #+END_SRC

   Note that 'playbook_directory' is special, as it
   allows you to define the directory all of your
   Ansible playbooks can be found. You can also specify
   this anywhere you can use the configuration clause,
   so you may set up different playbook directories for
   different playbook collections.

   #+BEGIN_SRC ruby
   # sṕecific configuration for :development
   configuration do 
    stack :development
    krell_type "t2.small"
    servers 1
    rolling 3
    krell_disk_size 20
   end
   #+END_SRC
   
   The above shows Ansible variables for my
   specialiezd setup that is geared with work
   with AWS. You are free to specify any
   variables here, which will be injected into
   ansible-playbook through the '--extra-vars'
   parameter.

   Here is a group clause with a single book in it:

   #+BEGIN_SRC ruby
   # Groups are executed serially.
   group :first, "our very first group" do
     # Books within a group are executed in parallel,
     # and therefore must be independent of each other.
     book :nat, "nat.yml"
   end
   #+END_SRC

   Which issues the following command to Ansible
   (based on the earlier configuration):

   #+BEGIN_SRC bash
   ansible-playbook playbooks/nat.yml \
     --extra-vars "playbook_directory=playbooks stack=development krell_type=t2.small servers=1 rolling=3 krell_disk_size=20"
   #+END_SRC

   And if our group had more book entries, as in the second 
   example:

   #+BEGIN_SRC ruby
   group :second, "our second group" do
     book :rabbit, "rabbitmq_cluster.yml" do
       krell_type "t2.medium"
     end

     book :es_cluster, "elasticsearch_cluster.yml" do
       esver "1.7.4"
       cluster_name :es
       servers 3
       heapsize "2g"
       krell_type "t2.medium"
       krell_disk_size 200
     end
   end
   #+END_SRC

   Both the :rabbit and :es_cluster books would be executed
   in parallel.

*** Dividing up your specs in other PowerPlay files
    Ruby, the underlying language, give you a lot of
    things for "free", like allowing you to load other powerplay files,
    for example:

    #+BEGIN_SRC ruby
    load 'production.play'
    #+END_SRC

    We mention this here for those who may not be familiar with Ruby,
    but may wish to section off your specifications thusly.

    You don't really need to know any Ruby, but it could increase 
    the span of what you might want to do. To get a quick taste,
    please checkout [[https://www.ruby-lang.org/en/documentation/quickstart/][Ruby in 20 Minutes]].

    It is also possible to leverage Ruby's metaprogramming techniques to
    create templates for your specificaitons, but at some point, as time
    allows, I may directly support this in the DSL. Please let your
    wishes be known to me for this and any other feature you might want to 
    see.

*** Running Powerplay
    If you type 'powerplay' without parameters, you are greeted with:
    #+BEGIN_SRC doc
Commands:
  powerplay help [COMMAND]                                            # Describe available commands or one specific command
  powerplay play <script> -p, --play=[NAME|all] Which playbook shelf  # Run the powerplay script.
  powerplay ttys                                                      # list all the TMUX ptys on the current window.

Options:
  -v, [--verbose=[1|2|3]]
                           # Default: 0
    #+END_SRC
    Please use the help feature to explain the subcommands and options. We 
    shall be adding many more subcommands and options as our needs demands.
    If you like to see something here, please submit it as an issue on Github.

    And for an example of play help, (note that this may not be up-to-date, so
    please run 'powerplay help play' on your installe version!)
    #+BEGIN_SRC doc
powerplay help play
Usage:
  powerplay play [script] -p, --play, --power, --play=[NAME[ NAME2...]|all]

Options:
  -m, [--tmux=[WINDOWNUMBERopt]]                                                                                              #  Send output to all tmux panes in the current window, or the numeric window specified.
  -p, --play, --power, --play=[NAME[ NAME2...]|all]                                                                           # Which PowerPlay playbooks (as opposed to Ansible playbooks) to specifically execute.
  -g, [--group=[NAME[ NAME2...]|all]]                                                                                         #  Which groups to execute.
                                                                                                                              # Default: [:all]
  -c, [--congroups], [--no-congroups]                                                                                         # Run the groups themselves concurrently
  -b, [--book=[NAME[ NAME2...]|all]]                                                                                          # Which books to execute.
                                                                                                                              # Default: [:all]
  -u, [--dryrun], [--no-dryrun]                                                                                               # Dry run, do not actually execute.
  -x, --extra-vars, [--extra=<BOOKNAME|all>:"key1a=value1a key2a=value2a... " [BOOKNAME2:"key1b=value1b key2b=value2b... "]]  # Pass custom parameters directly to playbooks. You may either pass parameters to all playbooks or specific ones.
  -v, [--verbose=[1|2|3]]
                                                                                                                              # Default: 0

Description:
  Plays a PowerPlay script. The entries in the script, as specified inside of a group, are run in parallel by default.
    #+END_SRC

    There is a short-hand 'pp' command you may use
    that has the 'play' task as the default. So, for
    example, rather than having to type:

    #+begin_src bash
    powerplay play -p development ...
    #+end_src

    You can do instead:

    #+begin_src bash
    pp -p development ...
    #+end_src

    In all our examples, we will use the longer
    'powerplay' command, but you can easily
    substitute 'pp'.

*** Example .play Script
    To play around with the example .play script,
    Clone the [[https://github.com/flajann2/ansible-powerplay][Ansible Powerplay]] project locally:

    #+BEGIN_SRC bash
    git clone git@github.com:flajann2/ansible-powerplay.git
    #+END_SRC

    and go to the examples directory to find test.play.
    
*** Submitting your example .play scripts
    Please feel free to do pull requests of your
    scripts or submit them to me as Gist snippets
    and I will include them if they are good.

** Concurrency
   We offer a finely controllable concurency model in
   the DSL with groups. The short of it is that a group
   may be marked as :sync or :async. All contents of a
   :sync group shall be executed serially.  All
   contents of an :async group shall be executed
   concurrently.

   As you can now nest groups, and that each group is
   either synchronous or asynchronous, how these
   interact requires a bit of understanding as to how
   the sync and async job queing mechanism in PowerPlay
   actually works.

*** The Gory Details behind how :sync and :async
    Internally, we have two job queues, sync_jobs
    and async_jobs. We also have -- at least
    conceptually -- two run queues, sync_runs and
    async_runs, to reflect queues of currenly
    running jobs, or books.  A "job" or a "book"
    represent an actual Ansible Playbook being
    run, or waiting to be run.
    
    | enqueue    | deque and run 'queues' |
    |------------+------------------------|
    | sync_jobs  | sync_runs              |
    | async_jobs | async_runs             |

    As well, we have the following queuing
    rules. Please note that "iff" is the
    mathematical "iff", meaning "if and only if".

    | rule            | details     | behavior                                             |
    |-----------------+-------------+------------------------------------------------------|
    | enqueue         | async job   | iff sync_jobs is empty and all sync_runs completed   |
    |                 | sync job    | iff async_jobs is empty and all async_runs completed |
    | dequeue and run | async queue | grab everything and run it concurrently              |
    |                 | sync queue  | grab one at a time and run it until it completes     |

    Note that "dequeue and run" flips back and
    forth between working on the sync and async
    queues. Never both simultaneously.

**** Nested Groups
     You can appreicate that understanding the
     behavior and "interaction" of nested queues
     can get pretty hairy, but just keep in mind
     the rules above, as your nesting will
     rigorously adhere to the logic above, even
     as it descends into the queues. The group
     designation only directly affects its
     immediate jobs, or books. It does not
     directly affect the books in its nested
     children.

     To ensure that the groups are themselves
     executed synchronously if the parent
     group is synchronous, internally we insert
     :noop book types to ensure the algorithm
     behaves itself accordingly. Otherwise,
     two consecutive async groups would appear
     to come from one async group.

**** Implemention of the Execution Planning [authoritative]
     In actuality, what we do at the DSL processing
     level is decide whether or not a book is a sync
     book or async book. We generate the actual command
     line code at that point, and create a pair [:sync,
     book] or [:async, book] and push that into the
     planning queue, which is a FIFO queue.

     (Note the the following is conceptual. In
     actuality, the info is all inside the book
     object.)

     | book        | enqueue to FIFO planning_queue |
     |-------------+--------------------------------|
     | sync group  | [:sync, bash string]           |
     | async group | [:async, bash string]          |
     | naked       | [:sync, bash string]           |

     We determine what execution planning a book gets
     by its immediate grouping. A group's default is
     :async.  Naked books are :sync by default. We do
     this to be intuitive about how things work in the
     DSL. You should explicitely have to specify what's
     going to be async, since that is the "more
     dangerous" mode.

     | dequeue from FIFO     | action                                                                                            |
     |-----------------------+---------------------------------------------------------------------------------------------------|
     | [:sync, bash string]  | join all entries in async_run_queue, clear that queue, and then execute and join bash string task |
     | [:async, bash string] | execute and enqueue to async_run_queue                                                            |
     |                       |                                                                                                   |

     This simplifies the algorithm and makes it easier
     to understand, and should result in a more
     intuitive grasp on how to write the PowerPlay.

**** TODO Scenarios

** Contributing to ansible-powerplay
   Your parcipitation is welcome, and I will
   respond to your pull requests in a timely
   fashion as long as I am not pulling an "Atlas"
   at my current job! lol
 
   + Check out the latest master to make sure the feature hasn't been implemented or the bug hasn't been fixed yet.
   + Check out the issue tracker to make sure someone already hasn't requested it and/or contributed it.
   + Fork the project.
   + Start a feature/bugfix branch.
   + Commit and push until you are happy with your contribution.
   + Make sure to add tests for it. This is important so I don't break it in a future version unintentionally.
   + Please try not to mess with the Rakefile, version, or history. If you want to have your own version, or is otherwise necessary, that is fine, but please isolate to its own commit so I can cherry-pick around it.

** Copyright
   Copyright (c) 2016 Fred Mitchell. See
   LICENSE.txt for further details.

** The Junkyard
   This area should be ignored, just a place
   for me to keep old snippets of code and other
   notes that will be of relevance to no one else.
*** Old execution planning model
    #+begin_src ruby
    # old code and will be deleted
    playbooks do |pname, playbook|
      group_threads = []
          puts "PLAYBOOK #{pname} (group=#{Play::clopts[:group]}) -->"
          groups playbook do |group|
            tg = nil
            group_threads << (tg = Thread.new {
                                puts "    GROUP #{group.type} (book=#{bucher}, cg=#{congroups}) -->"
                                book_threads = []
                                errors = []
                                group.books.each { |book| get_book_apcmd(book, bucher, book_threads, errors) }
                                book_threads.each{ |t| t.join }
                                unless errors.empty?
                                  errors.each do |yaml, cmd, txt|
                                    puts '=' * 30
                                    puts ('*' * 10) + ' ' + yaml
                                    puts txt
                                    puts '-' * 30
                                    puts cmd
                                  end
                                  exit 10
                                end
                              })
            # Always wait here unless we're concurrent
            group_threads.join unless congroups
          end
          group_threads.each{ |t| t.join }
        end
    #+end_src