README.org in org-converge-0.0.9 vs README.org in org-converge-0.0.10
- old
+ new
@@ -5,30 +5,31 @@
[[https://secure.travis-ci.org/wallyqs/org-converge.png?branch=master]]
** Description
-This attempts to be an experiment of using Org mode syntax to
+This is an experiment of using Org mode to
create documentable reproducible runs, borrowing some ideas
of what is possible to do with tools like =chef-solo=,
=rake=, =foreman=, etc...
** Installing
: gem install org-converge
** Motivation
-The Org babel syntax has proven to be flexible enough to writing
-/reproducible research/ papers. Then, I believe that configuring and setting up
-a server for example is something that could be also be done using
-the same syntax, given that /converging/ the configuration is something
-that one ought to be able to reproduce.
+[[http://orgmode.org/worg/org-contrib/babel/Org Babel][Org Mode]] has proven to be flexible enough to write
+[[http://www.jstatsoft.org/v46/i03][reproducible research papers]].
+Then, I believe that configuring and setting up
+a server for example, is something that could also be done using
+the same approach, given that /converging/ the configuration
+is a kind of run that one ought to be able to reproduce.
** Usage
-To run the blocks in parallel...
+To run the blocks in parallel:
#+begin_src sh
org-converge path/to/setup-file.org --runmode=parallel
#+end_src
@@ -43,37 +44,146 @@
#+begin_src sh
org-converge path/to/setup-file.org --runmode=chained
#+end_src
+In case the blocks have results blocks, it is possible to run
+the blocks and matched against the results with the ~org-spec~ helper command:
+
+#+begin_src sh
+org-spec path/to/file-spec.org
+#+end_src
+
*** Other commands available
: org-run # alias for org-converge
: org-tangle # just tangles the contents without running the blocks
-** Quick example
+** How it works
+Org Converge uses a liberally extended version of Org Babel
+features in order to give support for converging the configuration
+of a server, and the [[https://github.com/wallyqs/org-ruby][Ruby implementation of the Org mode parser]]
+which makes integrating with useful Ruby tools like foreman, rake, rspec and ohai more straightforward.
+
+For example, using Org Babel we can easily spread config
+files on a server by writing the following on a ~server.org~ file.
+
+#+begin_src sh
+,#+begin_src yaml :tangle /etc/component.yml
+multitenant: false
+status_port: 10004
+,#+end_src
+#+end_src
+
+And then configure it by running it as follows, (considering we have
+the correct permissions for tangling at =/etc/component.yml=):
+
+#+begin_src sh
+sudo org-converge server.org
+#+end_src
+
+Next, let's say that we no only one want to set the configured templates,
+but that we also want to install some packages. In that case, we
+should be able to do the following:
+(Note: Currently, only named blocks would be run)
+
+#+begin_src sh
+,** Configuring the component
+
+,#+begin_src yaml :tangle /etc/component.yml
+multitenant: false
+status_port: 10004
+,#+end_src
+
+,** Installing the dependencies
+
+Need the following so that ~bundle install~ can compile
+the native extensions correctly.
+
+# Giving the block a name would make it run
+
+,#+name: build_essentials
+,#+begin_src sh
+apt-get install build-essentials -y
+,#+end_src
+
+Then the following should work:
+
+,#+name: bundle_install
+,#+begin_src sh
+cd project_path
+bundle install
+,#+end_src
+#+end_src
+
+Since we are using Org mode syntax, it is possible to reuse this setup file by including it.
+
+#+begin_src sh
+,#+TITLE: Another setup
+
+Include the code blocks from the server into this:
+,#+include: "server.org"
+
+,#+name: install_org_mode
+,#+begin_src sh
+apt-get install org-mode -y
+,#+end_src
+#+end_src
+
+#+end_src
+
+** Examples
+
+Currently there is support for the following kind of runs:
+
+- runs where the blocks need to run sequentially (~--runmode=sequential~) ::
+
+ Each code block is part of a step to be ran
+
+- runs where the blocks need to be run in parallel (~--runmode=parallel~) ::
+
+ One example of this is having what is supposed to be a distributed system running locally for development (where ~foreman~ would be used).
+
+- runs where the blocks need to be run in sequence according to defined dependencies (~--runmode=chained~) ::
+
+ Set of runs that are usually covered by using something like rake, make, etc...
+
+- runs where the blocks are run and matched against the expected results for testing (~--runmode=spec~) ::
+
+ Each block is run and there is an assertion to check whether the contents in ~#+RESULTS~ block match
+
+Besides being able to specify which kind of run to use through an option, it is also possible
+to define this within the Org mode file itself as an in buffer setting:
+
+#+begin_src org
+ ,#+TITLE: Defining the runmode as an in buffer setting
+ ,#+runmode: sequential
+#+end_src
+
+*** Parallel runs
+
The following is an example of running 3 processes
in parallel by defining them as code blocks from
an Org mode file:
#+begin_src sh
,#+TITLE: Running Org babel processes in parallel
-
+
,* Print with different languages
-
+
,#+name: hello_from_bash
,#+begin_src sh :shebang #!/bin/bash
while true; do echo "hello world from bash"; sleep 1; done
,#+end_src
-
+
,#+name: hello_from_ruby
,#+begin_src ruby :shebang #!/usr/local/bin/ruby
$stdout.sync = true
loop { puts "hello world from ruby" ; sleep 1 }
,#+end_src
-
+
,#+name: hello_from_python
,#+begin_src python :shebang #!/usr/bin/python
import time
import sys
for i in range(0,100):
@@ -84,78 +194,92 @@
#+end_src
We store this in a file named =hello.org= and then run it as follows:
#+begin_src sh
-org-converge hello.org
+org-run hello.org
#+end_src
This would produce an output similar to the following:
#+begin_src sh
[2014-05-04T19:23:40 +0900] Tangling 0 files...
[2014-05-04T19:23:40 +0900] Tangling succeeded!
-[2014-05-04T19:23:40 +0900] Tangling 3 scripts within directory: /Users/mariko/repos/org-converge/run...
+[2014-05-04T19:23:40 +0900] Tangling 3 scripts within directory: /Users/wallyqs/repos/org-converge/run...
[2014-05-04T19:23:40 +0900] Running code blocks now! (3 runnable blocks found in total)
[2014-05-04T19:23:40 +0900] hello_from_bash (4664) -- started with pid 4664
[2014-05-04T19:23:40 +0900] hello_from_ruby (4665) -- started with pid 4665
[2014-05-04T19:23:40 +0900] hello_from_python (4666) -- started with pid 4666
[2014-05-04T19:23:40 +0900] hello_from_bash (4664) -- hello world from bash
[2014-05-04T19:23:41 +0900] hello_from_ruby (4665) -- hello world from ruby
[2014-05-04T19:23:41 +0900] hello_from_python (4666) -- hello world from python
[2014-05-04T19:23:42 +0900] hello_from_ruby (4665) -- hello world from ruby
#+end_src
-** How it works
+Also possible to specify the name of the block to be ran:
-Org Converge uses an liberally extended version of Org Babel
-features in order to give support for converging the configuration
-of a server.
-
-For example, using Org Babel and macros we can easily spread config
-files on a server by writing the following on a ~server.org~ file.
-
#+begin_src sh
-,#+begin_src yaml :tangle /etc/component.yml
-multitenant: false
-status_port: 10004
-,#+end_src
+org-run hello.org --name from_ruby
#+end_src
-And then configure it by running it as follows, (considering we have
-the correct permissions for tangling at =/etc/component.yml=):
+*** Spec mode
+In case the Org mode file has a results block which represents the expected result,
+there is an ~org-spec~ command which can be useful to check whether there is change.
+For example, given the following file stored in ~test.org~:
+
#+begin_src sh
-sudo org-converge server.org
+ ,#+TITLE: Expected results example
+
+ ,#+name: hello
+ ,#+begin_src ruby :results output
+ 10.times do
+ puts "hola"
+ end
+ ,#+end_src
+
+ ,#+RESULTS: hello
+ ,#+begin_example
+ hola
+ hola
+ hola
+ hola
+ hola
+ hola
+ hola
+ hola
+ hola
+ hola
+ ,#+end_example
#+end_src
-Next, let's say that we no only one want to set the configured templates,
-but that we also want to install some packages. In that case, we
-should be able to do the following:
+We can be able to verify whether this is still correct by running ~org-spec test.org~
#+begin_src sh
-,* Configuring the component
+Checking results from 'hello' code block: OK
+#+end_src
-,#+begin_src yaml :tangle /etc/component.yml
-multitenant: false
-status_port: 10004
-,#+end_src
+As an example, let's say that the behavior of the original code block changed, and now says hello 5 times instead.
+In that case the output would be as follows:
-,* Installing the dependencies
-
-Need the following so that ~bundle install~ can compile
-the native extensions correctly.
-
-,#+begin_src sh
-apt-get install build-essentials -y
-,#+end_src
-
-Then the following should work:
-
-,#+begin_src sh
-cd project_path
-bundle install
-,#+end_src
+#+begin_src diff
+Checking results from 'hello' code block: DIFF
+@@ -1,11 +1,6 @@
+-hola
+-hola
+-hola
+-hola
+-hola
+-hola
+-hola
+-hola
+-hola
+-hola
++hello
++hello
++hello
++hello
++hello
#+end_src
** Contributing
The project is in very early development at this moment, but if you