README.md in cmds-0.2.4 vs README.md in cmds-0.2.5
- old
+ new
@@ -3,32 +3,46 @@
`Cmds` tries to make it easier to read, write and remember using shell commands in Ruby.
It treats generating shell the in a similar fashion to generating SQL or HTML.
-1. [Status](#status)
-3. [Real-World Examples](#real-world-examples)
-2. [Installation](#installation)
+Best read at
+<http://www.rubydoc.info/gems/cmds/>
+where the API doc links should work and you got a table and contents.
+
+
-----------------------------------------------------------------------------
Status
-----------------------------------------------------------------------------
Ya know, before you get too excited...
It's kinda starting to work. I'll be using it for stuff and seeing how it goes, but no promises until `1.0` of course.
-----------------------------------------------------------------------------
+License
+-----------------------------------------------------------------------------
+
+MIT
+
+
+-----------------------------------------------------------------------------
Real-World Examples
-----------------------------------------------------------------------------
+Or, "what's it look like?"...
+
- Instead of
```Ruby
- `psql -U #{ db_config['username'] || ENV['USER'] } #{ db_config['database']} < #{ filepath.shellescape }`
+ `psql \
+ --username=#{ (db_config['username'] || ENV['USER']).shellescape } \
+ #{ db_config['database'].shellescape } \
+ < #{ filepath.shellescape }`
```
write
```Ruby
@@ -44,338 +58,455 @@
```bash
psql --username=nrser that_db < ./some/file/path
```
+ Cmds takes care of shell escaping for you.
+
- Instead of
```Ruby
- `aws s3 sync s3://#{ PROD_APP_NAME } #{ s3_path.shellescape }`
+ `PGPASSWORD=#{ config[:password].shellescape } \
+ pg_dump \
+ --username=#{ config[:username].shellescape } \
+ --host=#{ config[:host].shellescape } \
+ --port=#{ config[:port].shellescape } \
+ #{ config[:database].shellescape } \
+ > #{ filepath.shellescape }`
```
- write
+ which can be really hard to pick out what's going on from a quick glance, write
```Ruby
- Cmds 'aws s3 sync %{uri} %{path}',
- uri: "s3://#{ PROD_APP_NAME }",
- path: s3_path
+ Cmds.new(
+ 'pg_dump %{opts} %{database}',
+ kwds: {
+ opts: {
+ username: config[:username],
+ host: config[:host],
+ port: config[:port],
+ },
+ database: config[:database],
+ },
+ env: {
+ PGPASSWORD: config[:password],
+ },
+ ).stream! { |io| io.out = filename }
```
-
-
-- Instead of
- ```Ruby
- `PGPASSWORD=#{ config[:password].shellescape } pg_dump -U #{ config[:username].shellescape } -h #{ config[:host].shellescape } -p #{ config[:port] } #{ config[:database].shellescape } > #{ filepath.shellescape }`
- ```
+ I find it much easier to see what's going on their quickly.
- write
+ Again, with some additional comments and examples:
```Ruby
- Cmds 'PGPASSWORD=%{password} pg_dump %{opts} %{database} > %{filepath}',
- password: config[:password],
- database: config[:database],
- filepath: filepath,
- opts: {
- username: config[:username],
- host: config[:host],
- port: config[:port],
- }
+ # We're going to instantiate a new {Cmds} object this time, because we're
+ # not just providing values for the string template, we're specifying an
+ # environment variable for the child process too.
+ #
+ cmd = Cmds.new(
+ # The string template to use.
+ 'pg_dump %{opts} %{database}',
+ kwds: {
+ # Hashes will automatically be expanded to CLI options. By default,
+ # we use `--name=VALUE` format for long ones and `-n VALUE` for short,
+ # but it's configurable.
+ opts: {
+ username: config[:username],
+ host: config[:host],
+ port: config[:port],
+ },
+ # As mentioned above, everything is shell escaped automatically
+ database: config[:database],
+ },
+ # Pass environment as it's own Hash. There are options for how it is
+ # provided to the child process as well.
+ env: {
+ # Symbol keys are fine, we'll take care of that for you
+ PGPASSWORD: config[:password],
+ },
+ )
+
+ # Take a look!
+ cmd.prepare
+ # => "PGPASSWORD=shhh\\! pg_dump --host=localhost --port=5432 --username=nrser blah"
+
+ # Now stream it. the `!` means raise if the exit code is not 0
+ exit_code = cmd.stream! { |io|
+ # We use the block to configure I/O. Here we send the standard output to
+ # a file, which can be a String, Pathname or IO object
+ io.out = filename
+ }
```
+
-----------------------------------------------------------------------------
Installation
-----------------------------------------------------------------------------
Add this line to your application's `Gemfile`:
-```
-gem 'cmds'
-```
+ gem 'cmds'
And then execute:
-```
-$ bundle
-```
+ bundle install
-Or install it yourself as:
+Or install it globally with:
-```
-$ gem install cmds
-```
+ gem install cmds
-## architecture
-Cmds is based around a central `Cmds` class that takes a template for the command and a few options and operates by either wrapping the results in a `Cmds::Result` instance or streaming the results to `IO` objects or handler blocks. the Cmds` `augmented with a health helping of connivence methods for creating and executing a `Cmds` instance in common ways.
+-----------------------------------------------------------------------------
+Overview
+-----------------------------------------------------------------------------
-### constructor
+Cmds is based around a central {Cmds} class that takes a template for the command and a few options and operates by either wrapping the results in a {Cmds::Result} instance or streaming the results to `IO` objects or handler blocks.
-the `Cmds` constructor looks like
-```
-Cmds(template:String, opts:Hash)
-```
+-----------------------------------------------------------------------------
+Features
+-----------------------------------------------------------------------------
-a brief bit about the arguments:
+### Templates ###
-* `template`
- * a `String` template processed with ERB against positional and keyword arguments.
-* `opts`
- * `:args`
- * an `Array` of positional substitutions for the template.
- * assigned to `@args`.
- * defaults to an empty `Array`.
- * `:kwds`
- * a `Hash` of keyword substitutions for the template.
- * assigned to `@kwds`.
- * defaults to an empty `Hash`.
- * `:input`
- * a `String` to provide as standard input.
- * assigned to `@input`.
- * defaults to `nil`.
- * `:assert`
- * if this tests true, the execution of the command will raise an error on a nonzero exit status.
- * assigned to `@assert`.
- * defaults to `False`.
+#### ERB ####
-### execution
+Templates are processed with "[Embedded Ruby][]" (eRuby/ERB) using the [Erubis][] gem.
-you can provide three types of arguments when executing a command:
+[Embedded Ruby]: https://en.wikipedia.org/wiki/ERuby
+[Erubis]: http://www.kuwata-lab.com/erubis/
-1. positional arguments for substitution
-2. keyword arguments for substitution
-3. input to stdin
+For how it works check out
-all `Cmds` instance execution methods have the same form for accepting these:
+1. {Cmds::ERBContext}
+2. {Cmds::ShellERuby}
+3. {Cmds#render}
-1. positional arguments are provided in an optional array that must be the first argument:
+******************************************************************************
+
+
+##### Positional Values from `args` #####
+
+1. Use the `args` array made available in the templates with entry indexes.
- `Cmds "cp <%= arg %> <%= arg %>", [src_path, dest_path]`
+ Example when constructing:
- note that the arguments need to be enclosed in square braces. Cmds does **NOT** use \*splat for positional arguments because it would make a `Hash` final parameter ambiguous.
+ ```ruby
+ Cmds.new(
+ 'cp <%= args[0] %> <%= args[1] %>',
+ args: [
+ 'source.txt',
+ 'dest.txt',
+ ],
+ ).prepare
+ # => "cp source.txt dest.txt"
+ ```
-2. keyword arguments are provided as optional hash that must be the last argument:
+ This will raise an error if it's called after using the last positional argument, but will not complain if all positional arguments are not used.
- `Cmds "cp <%= src %> <%= dest %>", src: src_path, dest: dest_path`
+2. Use the `arg` method made available in the templates to get the next positional arg.
- in this case, curly braces are not required since Ruby turns the trailing keywords into a `Hash` provided as the last argument (or second-to-last argument in the case of a block included in the method signature).
+ Example when using "sugar" methods that take `args` as the single-splat (`*args`):
-3. input and output is handled with blocks:
-
- `Cmds(“wc -l”){ “one\ntwo\nthree\n” }
-
- Cmds.stream './test/tick.rb <%= times %>', times: times do |io|
- io.on_out do |line|
- # do something with the output line
- end
-
- io.on_err do |line|
- # do something with the error line
- end
- end`
+ ```ruby
+ Cmds.prepare 'cp <%= arg %> <%= arg %>',
+ 'source.txt',
+ 'dest.txt'
+ # => "cp source.txt dest.txt"
+ ```
+******************************************************************************
-## templates
+##### Keyword Values from `kwds` #####
-command templates are processed with [eRuby](https://en.wikipedia.org/wiki/ERuby), which many people know as [ERB](http://ruby-doc.org/stdlib-2.2.2/libdoc/erb/rdoc/ERB.html). you may know ERB from [Rails](http://guides.rubyonrails.org/layouts_and_rendering.html).
+Just use the key as the method name.
-actually, Cmds uses [Erubis](http://www.kuwata-lab.com/erubis/). which is the same thing Rails uses; calm down.
+When constructing:
-this takes care of a few things:
+```ruby
+Cmds.new(
+ 'cp <%= src %> <%= dest %>',
+ kwds: {
+ src: 'source.txt',
+ dest: 'dest.txt',
+ },
+).prepare
+# => "cp source.txt dest.txt"
+```
-1. automatically shell escape values substituted into templates with [`Shellwords.escape`](http://ruby-doc.org/stdlib-2.2.2/libdoc/shellwords/rdoc/Shellwords.html#method-c-escape). it doesn't always do the prettiest job, but `Shellwords.escape` is part of Ruby's standard library and seems to work pretty well.
-2. allow for fairly nice and readable logical structures like `if` / `else` in the command template. you've probably built html like this at some point. of course, the full power of Ruby is also available, though you probably won't find yourself needing much beyond some simple control structures.
+When using "sugar" methods that take `kwds` as the double-splat (`**kwds`):
-## substitutions
+```ruby
+Cmds.prepare 'cp <%= src %> <%= dest %>',
+ src: 'source.txt',
+ dest: 'dest.txt'
+# => "cp source.txt dest.txt"
+```
-substitutions can be positional, keyword, or both.
-### positional
+###### Key Names to Avoid ######
-positional arguments can be substituted in order using the `arg` method call:
+If possible, avoid naming your keys:
-```
-Cmds.sub "psql <%= arg %> <%= arg %> < <%= arg %>", [
- {
- username: "bingo bob",
- host: "localhost",
- port: 12345,
- },
- "blah",
- "/where ever/it/is.psql",
-]
-# => 'psql --host=localhost --port=12345 --username=bingo\ bob blah < /where\ ever/it/is.psql'
-```
+- `arg`
+- `args`
+- `initialize`
+- `get_binding`
+- `method_missing`
-internally this translates to calling `@args.fetch(@arg_index)` and increments `@arg_index` by 1.
+If you must name them those things, don't expect to be able to access them as shown above; use `<%= @kwds[key] %>`.
-this will raise an error if it's called after using the last positional argument, but will not complain if all positional arguments are not used. this prevents using a keyword arguments named `arg` without accessing the keywords hash directly.
-the arguments may also be accessed directly though the bound class's `@args` instance variable:
+###### Keys That Might Not Be There ######
+Normally, if you try to interpolate a key that doesn't exist you will get a `KeyError`:
+
+```ruby
+Cmds.prepare "blah <%= maybe %> <%= arg %>", "value"
+# KeyError: couldn't find keys :maybe or "maybe" in keywords {}
```
-Cmds.sub "psql <%= @args[2] %> <%= @args[0] %> < <%= @args[1] %>", [
- "blah",
- "/where ever/it/is.psql",
- {
- username: "bingo bob",
- host: "localhost",
- port: 12345,
- },
-]
-# => 'psql --host=localhost --port=12345 --username=bingo\ bob blah < /where\ ever/it/is.psql'
-```
-note that `@args` is a standard Ruby array and will simply return `nil` if there is no value at that index (though you can use `args.fetch(i)` to get the same behavior as the `arg` method with a specific index `i`).
+I like a lot this better than just silently omitting the value, but sometimes you know that they key might not be set and want to receive `nil` if it's not.
-### keyword
+In this case, append `?` to the key name (which is a method call in this case) and you will get `nil` if it's not set:
-keyword arguments can be accessed by making a method call with their key:
+```ruby
+Cmds.prepare "blah <%= maybe? %> <%= arg %>", "value"
+# => "blah value"
+```
+```ruby
+Cmds.prepare "blah <%= maybe? %> <%= arg %>", "value", maybe: "yes"
+# => "blah yes value"
```
-Cmds.sub "psql <%= opts %> <%= database %> < <%= filepath %>",
- [],
- database: "blah",
- filepath: "/where ever/it/is.psql",
- opts: {
- username: "bingo bob",
- host: "localhost",
- port: 12345,
- }
-# => 'psql --host=localhost --port=12345 --username=bingo\ bob blah < /where\ ever/it/is.psql'
-```
-this translates to a call of `@kwds.fetch(key)`, which will raise an error if `key` isn't present.
+******************************************************************************
-there are four key names that may not be accessed this way due to method definition on the context object:
-* `arg` (see above)
-* `initialize`
-* `get_binding`
-* `method_missing`
+##### Shell Escaping #####
-though keys with those names may be accessed directly via `@kwds.fetch(key)` and the like.
+Cmds automatically shell-escapes values it interpolates into templates by passing them through the Ruby standard libray's [Shellwords.escape][].
-to test for a key's presence or optionally include a value, append `?` to the method name:
+[Shellwords.escape]: http://ruby-doc.org/stdlib/libdoc/shellwords/rdoc/Shellwords.html#method-c-escape
+```ruby
+Cmds.prepare "cp <%= src %> <%= dest %>",
+ src: "source.txt",
+ dest: "path with spaces.txt"
+=> "cp source.txt path\\ with\\ spaces.txt"
```
-c = Cmds.new <<-BLOCK
- defaults
- <% if current_host? %>
- -currentHost <%= current_host %>
- <% end %>
- export <%= domain %> <%= filepath %>
-BLOCK
-c.call domain: 'com.nrser.blah', filepath: '/tmp/export.plist'
-# defaults export com.nrser.blah /tmp/export.plist
+It doesn't always do the prettiest job, but it's part of the standard library and seems to work pretty well... shell escaping is a messy and complicated topic (escaping for *which* shell?!), so going with the built-in solution seems reasonable for the moment, though I do hate all those backslashes... they're a pain to read.
-c.call current_host: 'xyz', domain: 'com.nrser.blah', filepath: '/tmp/export.plist'
-# defaults -currentHost xyz export com.nrser.blah /tmp/export.plist
-```
-### both
+###### Raw Interpolation ######
-both positional and keyword substitutions may be provided:
+You can render a raw string with `<%== %>`.
+To see the difference with regard to the previous example (which would break the `cp` command in question):
+
+```ruby
+Cmds.prepare "cp <%= src %> <%== dest %>",
+ src: "source.txt",
+ dest: "path with spaces.txt"
+=> "cp source.txt path with spaces.txt"
```
-Cmds.sub "psql <%= opts %> <%= arg %> < <%= filepath %>",
- ["blah"],
- filepath: "/where ever/it/is.psql",
- opts: {
- username: "bingo bob",
- host: "localhost",
- port: 12345,
- }
-# => 'psql --host=localhost --port=12345 --username=bingo\ bob blah < /where\ ever/it/is.psql'
-```
-this might be useful if you have a simple positional command like
+And a way it make a little more sense:
+```ruby
+Cmds.prepare "<%== bin %> <%= *args %>",
+ 'blah',
+ 'boo!',
+ bin: '/usr/bin/env echo'
+=> "/usr/bin/env echo blah boo\\!"
```
-Cmds "blah <%= arg %>", ["value"]
-```
-and you want to quickly add in some optional value
+******************************************************************************
+
+##### Splatting (`*`) To Render Multiple Shell Tokens #####
+
+Render multiple shell tokens (individual strings the shell picks up - basically, each one is an entry in `ARGV` for the child process) in one expression tag by prefixing the value with `*`:
+
+```ruby
+Cmds.prepare '<%= *exe %> <%= cmd %> <%= opts %> <%= *args %>',
+ 'x', 'y', # <= these are the `args`
+ exe: ['/usr/bin/env', 'blah'],
+ cmd: 'do-stuff',
+ opts: {
+ really: true,
+ 'some-setting': 'dat-value',
+ }
+# => "/usr/bin/env blah do-stuff --really --some-setting=dat-value x y"
```
-Cmds "blah <%= maybe? %> <%= arg %>", ["value"]
-Cmds "blah <%= maybe? %> <%= arg %>", ["value"], maybe: "yes!"
-```
-************************************************************************
+`ARGV` tokenization by the shell would look like:
+```ruby
+[
+ '/usr/bin/env',
+ 'blah',
+ 'do-stuff',
+ '--really',
+ '--some-setting=dat-value',
+ 'x',
+ 'y',
+]
+```
-### Shortcuts
+- Compare to *without* splats:
+
+ ```ruby
+ Cmds.prepare '<%= exe %> <%= cmd %> <%= opts %> <%= args %>',
+ 'x', 'y', # <= these are the `args`
+ exe: ['/usr/bin/env', 'blah'],
+ cmd: 'do-stuff',
+ opts: {
+ really: true,
+ 'some-setting': 'dat-value',
+ }
+ # => "/usr/bin/env,blah do-stuff --really --some-setting=dat-value x,y"
+ ```
+
+ Which is probably *not* what you were going for... it would produce an `ARGV` something like:
+
+ ```ruby
+ [
+ '/usr/bin/env,blah',
+ 'do-stuff',
+ '--really',
+ '--some-setting=dat-value',
+ 'x,y',
+ ]
+ ```
-`Cmds` has (limited, custom) support for [printf][]-style shortcuts.
+You can of course use "splatting" together with slicing or mapping or whatever.
-[printf]: https://en.wikipedia.org/wiki/Printf_format_string
+******************************************************************************
-**positional**
+##### Logic #####
-`%s` is replaced with `<%= arg %>`.
+All of ERB is available to you. I've tried to put in features and options that make it largely unnecessary, but if you've got a weird or complicated case, or if you just like the HTML/Rails-esque templating style, it's there for you:
-so
+```ruby
+cmd = Cmds.new <<-END
+ <% if use_bin_env %>
+ /usr/bin/env
+ <% end %>
+
+ docker build .
+ -t <%= tag %>
+
+ <% if file %>
+ --file <%= file %>
+ <% end %>
+
+ <% build_args.each do |key, value| %>
+ --build-arg <%= key %>=<%= value %>
+ <% end %>
+
+ <% if yarn_cache %>
+ --build-arg yarn_cache_file=<%= yarn_cache_file %>
+ <% end %>
+END
+cmd.prepare(
+ use_bin_env: true,
+ tag: 'nrser/blah:latest',
+ file: './prod.Dockerfile',
+ build_args: {
+ yarn_version: '1.3.2',
+ },
+ yarn_cache: true,
+ yarn_cache_file: './yarn-cache.tgz',
+)
+# => "/usr/bin/env docker build . -t nrser/blah:latest
+# --file ./prod.Dockerfile --build-arg yarn_version=1.3.2
+# --build-arg yarn_cache_file=./yarn-cache.tgz"
+# (Line-breaks added for readability; output is one line)
```
-Cmds.sub "./test/echo_cmd.rb %s", ["hello world!"]
-```
-is the same as
+******************************************************************************
-```
-Cmds "./test/echo_cmd.rb <%= arg %>", ["hello world!"]
-```
-**keyword**
+#### `printf`-Style Short-Hand (`%s`, `%{key}`, `%<key>s`)
-`%{key}` and `%<key>s` are replaced with `<%= key %>`, and `%{key?}` and `%<key?>s` are replaced with `<%= key? %>` for optional keywords.
+Cmds also supports a [printf][]-style short-hand. Sort-of.
-so
+[printf]: https://en.wikipedia.org/wiki/Printf_format_string
-```
-Cmds "./test/echo_cmd.rb %{key}", key: "hello world!"
-```
+It's a clumsy hack from when I was first writing this library, and I've pretty moved to using the ERB-style, but there are still some examples that use it, and I guess it still works (to whatever extent it ever really did), so it's probably good to mention it.
-and
+It pretty much just replaces some special patterns with their ERB-equivalent via the {Cmds.replace_shortcuts} method before moving on to ERB processing:
-```
-Cmds "./test/echo_cmd.rb %<key>s", key: "hello world!"
-```
+1. `%s` => `<%= arg %>`
+2. `%{key}` => `<%= key %>`
+3. `%{key?}` => `<%= key? %>`
+4. `%<key>s` => `<%= key %>`
+5. `%<key?>s` => `<%= key? %>`
-are the same is
+And the escaping versions, where you can put anothe `%` in front to get the literal intead of the subsitution:
-```
-Cmds "./test/echo_cmd.rb <%= key %>", key: "hello world!"
-```
+1. `%%s` => `%s`
+2. `%%{key}` => `%{key}`
+3. `%%{key?}` => `%{key?}`
+4. `%%<key>s` => `%<key>s`
+5. `%%<key?>s` => `%<key?>s`
-**escaping**
+That's it. No `printf` formatting beyond besides `s` (string).
-strings that would be replaced as shortcuts can be escaped by adding one more `%` to the front of them:
-```
-Cmds.sub "%%s" # => "%s"
-Cmds.sub "%%%<key>s" # => "%%<key>s"
-```
+-----------------------------------------------------------------------------
+Old docs I haven't cleaned up yet...
+-----------------------------------------------------------------------------
-note that unlike `sprintf`, which has a much more general syntax, this is only necessary for patterns that exactly match a shortcut, not `%` in general:
+### execution
-```
-Cmds.sub "50%" # => "50%"
-```
+you can provide three types of arguments when executing a command:
+1. positional arguments for substitution
+2. keyword arguments for substitution
+3. input to stdin
+all `Cmds` instance execution methods have the same form for accepting these:
-## reuse commands
+1. positional arguments are provided in an optional array that must be the first argument:
+
+ `Cmds "cp <%= arg %> <%= arg %>", [src_path, dest_path]`
+
+ note that the arguments need to be enclosed in square braces. Cmds does **NOT** use \*splat for positional arguments because it would make a `Hash` final parameter ambiguous.
+
+2. keyword arguments are provided as optional hash that must be the last argument:
+
+ `Cmds "cp <%= src %> <%= dest %>", src: src_path, dest: dest_path`
+
+ in this case, curly braces are not required since Ruby turns the trailing keywords into a `Hash` provided as the last argument (or second-to-last argument in the case of a block included in the method signature).
+
+3. input and output is handled with blocks:
+
+ `Cmds(“wc -l”){ “one\ntwo\nthree\n” }
+
+ Cmds.stream './test/tick.rb <%= times %>', times: times do |io|
+ io.on_out do |line|
+ # do something with the output line
+ end
+
+ io.on_err do |line|
+ # do something with the error line
+ end
+ end`
+
+### Reuse Commands
+
```
playbook = Cmds.new "ansible-playbook -i %{inventory} %{playbook}"
playbook.call inventory: "./hosts", playbook: "blah.yml"
```
@@ -391,13 +522,12 @@
# run setup.yml on the production hosts
prod_playbook.call playbook: "setup.yml"
```
+### defaults
-## defaults
-
NEEDS TEST
can be accomplished with reuse and currying stuff
```
@@ -409,25 +539,23 @@
# run setup.yml on the production hosts
prod_playbook.call playbook: "setup.yml", inventory: "inventory/prod"
```
+### input
-## input
-
```
c = Cmds.new("wc", input: "blah blah blah).call
```
+### future..?
-## future..?
+#### exec
-### exec
-
want to be able to use to exec commands
-### formatters
+#### formatters
kinda like `sprintf` formatters or string escape helpers in Rails, they would be exposed as functions in ERB and as format characters in the shorthand versions:
```
Cmds "blah <%= j obj %>", obj: {x: 1}