README.org in ansible-powerplay-0.2.4 vs README.org in ansible-powerplay-1.0.2

@@ -1,77 +1,138 @@
 * Ansible Powerplay
 
-  Powerplay allows you to run multiple Ansible playbooks in
-  parallel. Depending on how you organize your playbooks,
-  this can be a solid win. I basically before this had been
-  doing a playbook with multiple includes for other playbooks
-  representing different servers in our stack. Playbook launching
-  of playbooks is slow and very serial.
+  #+ATTR_HTML: title="Join the chat at https://gitter.im/flajann2/ansible-powerplay"
+  [[https://gitter.im/flajann2/ansible-powerplay?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge][file:https://badges.gitter.im/flajann2/ansible-powerplay.svg]]
 
-  Basically, the playbooks are all contained, 
-  so no interdependencies. And in my case, running in the
-  cloud, so no reason why they can't be running in parallel
+  Powerplay allows you to run multiple Ansible
+  playbooks in parallel. Depending on how you organize
+  your playbooks, this can be a solid win. I basically
+  before this had been doing a playbook with multiple
+  includes for other playbooks representing different
+  servers in our stack. Playbook launching of playbooks
+  is slow and very serial.
 
-  Powerplay allows you to specify vars common
-  to all playbooks, and also vars specific
-  to some playbooks so by which you can
-  make your setup very DRY.
+  Basically, the playbooks are all contained, so no
+  interdependencies. And in my case, running in the
+  cloud, so no reason why they can't be running in
+  parallel
 
-  All the Ansible playbooks are executed in seperate processes,
-  and thus avoiding a number of the "side effects" you would
-  normally encounter with running multiple playbooks with
-  Ansible includes.
+  Powerplay allows you to specify vars common to all
+  playbooks, and also vars specific to some playbooks
+  so by which you can make your setup very DRY.
 
+  All the Ansible playbooks are executed in seperate
+  processes, and thus avoiding a number of the "side
+  effects" you would normally encounter with running
+  multiple playbooks with Ansible includes.
+
   For example, here is Powerplay integrated with tmux:
   #+CAPTION: Powerplay writing to tmux panes, one pane per playbook.
   #+NAME: Powerplay Example
   [[./examples/powerplay_screenshot.jpeg]]
 
 ** Release Notes
-   Please see [[RELEASE_NOTES.org][Release Notes]]
+   Please see [[RELEASE_NOTES.org][Release Notes]].
+
+   The version 1.x releases have a new DSL
+   which may be slightly incompatable with the older 0.x DSL.
+   There will be a lapse between this release and the
+   documentaiton of the differences.
+
+   Also, the capture of the output from ansible-powerplay is 
+   handled a bit more intelligently. If you do not specify the
+   --tmux (or -m) option, all output is now currently captured
+   by powerplay and redumped to the console.
+
+   Because you may still want to see the color from ansible-powerplay,
+   you can alter ansible.cfg and add the following line:
+
+   + force_color = 1
+
 ** Features and Cavets
 *** Integration with TMUX
-    When running multiple Ansible Playbooks concurrently,
-    one would like to be able to see the output of each
-    in a reasonable manner. To faciliate this in this
-    initial realse, we shall make heavy use of TMUX panes
-    to dump the output.
+    When running multiple Ansible Playbooks
+    concurrently, one would like to be able to see the
+    output of each in a reasonable manner. To faciliate
+    this in this initial realse, we shall make heavy
+    use of TMUX panes to dump the output.
 
     So basically, you need as many panes as you have
-    concurrent Ansible Playbooks in this initial release. In 
-    subsequent releases, Curses will be directly
-    leveraged to create "tabs" for the multiple output
-    streams. We may even do this, still, through TMUX.
+    concurrent Ansible Playbooks in this initial
+    release. In subsequent releases, Curses will be
+    directly leveraged to create "tabs" for the
+    multiple output streams. We may even do this,
+    still, through TMUX.
 
-    Your input on this is strongly encouarged. We will not
-    be supporting Screen at all. Sorry.
+    Your input on this is strongly encouarged. We will
+    not be supporting Screen at all. Sorry.
 
 ** DSL Terminology & Documentation
+   Note that this is the DSL for version 1.x of
+   PowerPlay. For 0.x, please see those tags in
+   GitHub.
+
 *** DSL
     The DSL is straightforward as possible,
     simple and elegant to allow you to write 
     your Powerplays in a DRY manner.
+
+    For examples, please see the following:
+    | [[examples/stack.play][stack.play]]       | This is loaded by default, and you must be in your current directory     |
+    | [[examples/development.play][development.play]] | This is a fullblown Power Playbook for a hypothetical development stack. |
+    | [[examples/production.play][production.play]]  | This is a fullblown Power Playbook for a hypothetical production stack.  |
+    | [[examples/playbooks][playbooks]]        | Sample Ansible playbooks called by Powerplay.                            |
+
+    To run the powerplay example:
+
+    1. Install Ansible Powerplay
+       + gem install ansible-powerplay
+    2. Clone this project locally, then cd into the examples directory
+       + git clone https://github.com/flajann2/ansible-powerplay.git
+       + cd ansible-powerplay/examples
+    3. source ansible-paths and run Powerplay
+       + source ansible-paths.sh
+       + powerplay play -p development -v2
+
+    Note that I deliberately left a missing "elasticsearch.yml" so you
+    can see how Powerplay handles the errors.
+      
 **** configuration
      You can intersperse configuration blocks
      anywhere, and the expected nested scoping
      will take effect.
 **** playbooks
-     playbooks are a collection of groups, and
-     each group are normally executed serially. This will
+     playbooks are a collection of groups, and a group
+     defaults to async mode for its members.
+
+     Group are normally executed serially. This will
      allow you to organize your plays in an intelligent
      manner to deploy and manage resources and assets
      that may have to be done in a serial manner.
 **** group
-     A group is a collection of books that all execute
-     in parallel. Books are required to be independent of
-     each other.
+     A group is a collection of books or other groups
+     that all execute in parallel by default.
+     Books are required to be independent of 
+     each other. If they are not, you can set
+     them up to execute serially.
+
 **** book
      A book has a direct correspondence to an Ansible
      playbook, and will execute that Yaml file
      given the configuration variables as parameters.
 
      Here is where var inheritance becomes useful.
+     Note that all the configuration variables
+     set at the time the book is called are all
+     passed in as --extra-vars to Ansible Playbook.
+     The Playbook may not need all the vars passed
+     in, but care must be taken that no vars
+     are used in a different manner than expected.
+     We currently have no way of knowing which
+     vars are needed or not, and to specifiy that 
+     would make the syntax messy and loose some
+     of the advantages of var inheritance.
 
 ** Installation
    Easy installation. From command-line:
    #+BEGIN_SRC bash
    gem install ansible-powerplay
@@ -86,28 +147,28 @@
    Basically, cd to the root of your Ansible directory,
    and a .play file (see the example at: [[https://github.com/flajann2/ansible-powerplay/blob/master/examples/stack.play][stack.play]].)
 
    You can place a config clause either globally,
    inside of playbooks, inside of groups, and the
-   variable set up this way are inherited to the inner
-   clauses, thus allowing you to keep your specifications
-   DRYer.
+   variable set up this way are inherited to the
+   inner clauses, thus allowing you to keep your
+   specifications DRYer.
 
    For example:
    #+BEGIN_SRC ruby
    # This is a global system configuration
    configuration :system do
      playbook_directory "playbooks"
    end
    #+END_SRC
 
-   Note that 'playbook_directory' is special, as it allows
-   you to define the directory all of your Ansible playbooks
-   can be found. You can also specify this anywhere
-   you can use the configuration clause, so you
-   may set up different playbook directories for different
-   playbook collections.
+   Note that 'playbook_directory' is special, as it
+   allows you to define the directory all of your
+   Ansible playbooks can be found. You can also specify
+   this anywhere you can use the configuration clause,
+   so you may set up different playbook directories for
+   different playbook collections.
 
    #+BEGIN_SRC ruby
    # sṕecific configuration for :development
    configuration do 
     stack :development
@@ -116,14 +177,16 @@
     rolling 3
     krell_disk_size 20
    end
    #+END_SRC
    
-   The above shows Ansible variables for my specialiezd setup
-   that is geared with work with AWS. You are free to specify
-   any variables here, which will be injected into 
-   ansible-playbook through the '--extra-vars' parameter.
+   The above shows Ansible variables for my
+   specialiezd setup that is geared with work
+   with AWS. You are free to specify any
+   variables here, which will be injected into
+   ansible-playbook through the '--extra-vars'
+   parameter.
 
    Here is a group clause with a single book in it:
 
    #+BEGIN_SRC ruby
    # Groups are executed serially.
@@ -132,12 +195,12 @@
      # and therefore must be independent of each other.
      book :nat, "nat.yml"
    end
    #+END_SRC
 
-   Which issues the following command to Ansible (based on the
-   earlier configuration):
+   Which issues the following command to Ansible
+   (based on the earlier configuration):
 
    #+BEGIN_SRC bash
    ansible-playbook playbooks/nat.yml \
      --extra-vars "playbook_directory=playbooks stack=development krell_type=t2.small servers=1 rolling=3 krell_disk_size=20"
    #+END_SRC
@@ -225,11 +288,28 @@
 
 Description:
   Plays a PowerPlay script. The entries in the script, as specified inside of a group, are run in parallel by default.
     #+END_SRC
 
+    There is a short-hand 'pp' command you may use
+    that has the 'play' task as the default. So, for
+    example, rather than having to type:
 
+    #+begin_src bash
+    powerplay play -p development ...
+    #+end_src
+
+    You can do instead:
+
+    #+begin_src bash
+    pp -p development ...
+    #+end_src
+
+    In all our examples, we will use the longer
+    'powerplay' command, but you can easily
+    substitute 'pp'.
+
 *** Example .play Script
     To play around with the example .play script,
     Clone the [[https://github.com/flajann2/ansible-powerplay][Ansible Powerplay]] project locally:
 
     #+BEGIN_SRC bash
@@ -241,21 +321,159 @@
 *** Submitting your example .play scripts
     Please feel free to do pull requests of your
     scripts or submit them to me as Gist snippets
     and I will include them if they are good.
 
+** Concurrency
+   We offer a finely controllable concurency model in
+   the DSL with groups. The short of it is that a group
+   may be marked as :sync or :async. All contents of a
+   :sync group shall be executed serially.  All
+   contents of an :async group shall be executed
+   concurrently.
+
+   As you can now nest groups, and that each group is
+   either synchronous or asynchronous, how these
+   interact requires a bit of understanding as to how
+   the sync and async job queing mechanism in PowerPlay
+   actually works.
+
+*** The Gory Details behind how :sync and :async
+    Internally, we have two job queues, sync_jobs
+    and async_jobs. We also have -- at least
+    conceptually -- two run queues, sync_runs and
+    async_runs, to reflect queues of currenly
+    running jobs, or books.  A "job" or a "book"
+    represent an actual Ansible Playbook being
+    run, or waiting to be run.
+    
+    | enqueue    | deque and run 'queues' |
+    |------------+------------------------|
+    | sync_jobs  | sync_runs              |
+    | async_jobs | async_runs             |
+
+    As well, we have the following queuing
+    rules. Please note that "iff" is the
+    mathematical "iff", meaning "if and only if".
+
+    | rule            | details     | behavior                                             |
+    |-----------------+-------------+------------------------------------------------------|
+    | enqueue         | async job   | iff sync_jobs is empty and all sync_runs completed   |
+    |                 | sync job    | iff async_jobs is empty and all async_runs completed |
+    | dequeue and run | async queue | grab everything and run it concurrently              |
+    |                 | sync queue  | grab one at a time and run it until it completes     |
+
+    Note that "dequeue and run" flips back and
+    forth between working on the sync and async
+    queues. Never both simultaneously.
+
+**** Nested Groups
+     You can appreicate that understanding the
+     behavior and "interaction" of nested queues
+     can get pretty hairy, but just keep in mind
+     the rules above, as your nesting will
+     rigorously adhere to the logic above, even
+     as it descends into the queues. The group
+     designation only directly affects its
+     immediate jobs, or books. It does not
+     directly affect the books in its nested
+     children.
+
+     To ensure that the groups are themselves
+     executed synchronously if the parent
+     group is synchronous, internally we insert
+     :noop book types to ensure the algorithm
+     behaves itself accordingly. Otherwise,
+     two consecutive async groups would appear
+     to come from one async group.
+
+**** Implemention of the Execution Planning [authoritative]
+     In actuality, what we do at the DSL processing
+     level is decide whether or not a book is a sync
+     book or async book. We generate the actual command
+     line code at that point, and create a pair [:sync,
+     book] or [:async, book] and push that into the
+     planning queue, which is a FIFO queue.
+
+     (Note the the following is conceptual. In
+     actuality, the info is all inside the book
+     object.)
+
+     | book        | enqueue to FIFO planning_queue |
+     |-------------+--------------------------------|
+     | sync group  | [:sync, bash string]           |
+     | async group | [:async, bash string]          |
+     | naked       | [:sync, bash string]           |
+
+     We determine what execution planning a book gets
+     by its immediate grouping. A group's default is
+     :async.  Naked books are :sync by default. We do
+     this to be intuitive about how things work in the
+     DSL. You should explicitely have to specify what's
+     going to be async, since that is the "more
+     dangerous" mode.
+
+     | dequeue from FIFO     | action                                                                                            |
+     |-----------------------+---------------------------------------------------------------------------------------------------|
+     | [:sync, bash string]  | join all entries in async_run_queue, clear that queue, and then execute and join bash string task |
+     | [:async, bash string] | execute and enqueue to async_run_queue                                                            |
+     |                       |                                                                                                   |
+
+     This simplifies the algorithm and makes it easier
+     to understand, and should result in a more
+     intuitive grasp on how to write the PowerPlay.
+
+**** TODO Scenarios
+
 ** Contributing to ansible-powerplay
-   Your parcipitation is welcome, and I will respond to your
-   pull requests in a timely fashion as long as I am not
-   pulling an "Atlas" at my current job! lol
+   Your parcipitation is welcome, and I will
+   respond to your pull requests in a timely
+   fashion as long as I am not pulling an "Atlas"
+   at my current job! lol
  
    + Check out the latest master to make sure the feature hasn't been implemented or the bug hasn't been fixed yet.
    + Check out the issue tracker to make sure someone already hasn't requested it and/or contributed it.
    + Fork the project.
    + Start a feature/bugfix branch.
    + Commit and push until you are happy with your contribution.
    + Make sure to add tests for it. This is important so I don't break it in a future version unintentionally.
    + Please try not to mess with the Rakefile, version, or history. If you want to have your own version, or is otherwise necessary, that is fine, but please isolate to its own commit so I can cherry-pick around it.
 
 ** Copyright
-   Copyright (c) 2016 Fred Mitchell. See LICENSE.txt for
-   further details.
+   Copyright (c) 2016 Fred Mitchell. See
+   LICENSE.txt for further details.
+
+** The Junkyard
+   This area should be ignored, just a place
+   for me to keep old snippets of code and other
+   notes that will be of relevance to no one else.
+*** Old execution planning model
+    #+begin_src ruby
+    # old code and will be deleted
+    playbooks do |pname, playbook|
+      group_threads = []
+          puts "PLAYBOOK #{pname} (group=#{Play::clopts[:group]}) -->"
+          groups playbook do |group|
+            tg = nil
+            group_threads << (tg = Thread.new {
+                                puts "    GROUP #{group.type} (book=#{bucher}, cg=#{congroups}) -->"
+                                book_threads = []
+                                errors = []
+                                group.books.each { |book| get_book_apcmd(book, bucher, book_threads, errors) }
+                                book_threads.each{ |t| t.join }
+                                unless errors.empty?
+                                  errors.each do |yaml, cmd, txt|
+                                    puts '=' * 30
+                                    puts ('*' * 10) + ' ' + yaml
+                                    puts txt
+                                    puts '-' * 30
+                                    puts cmd
+                                  end
+                                  exit 10
+                                end
+                              })
+            # Always wait here unless we're concurrent
+            group_threads.join unless congroups
+          end
+          group_threads.each{ |t| t.join }
+        end
+    #+end_src