README.md in elasticity-6.0.5 vs README.md in elasticity-6.0.6
- old
+ new
@@ -1,10 +1,10 @@
[](http://badge.fury.io/rb/elasticity)
Elasticity provides programmatic access to Amazon's Elastic Map Reduce service. The aim is to conveniently abstract away the complex EMR REST API and make working with job flows more productive and more enjoyable.
-[](http://travis-ci.org/rslifka/elasticity) 2.0.x, 2.1.x, 2.2.x
+[](http://travis-ci.org/rslifka/elasticity) 2.0.x, 2.1.x, 2.2.x, 2.3.x
Elasticity provides two ways to access EMR:
* **Indirectly through a JobFlow-based API**. This README discusses the Elasticity API.
* **Directly through access to the EMR REST API**. The less-discussed hidden darkside... I use this to enable the Elasticity API. RubyDoc can be found at the RubyGems [auto-generated documentation site](http://rubydoc.info/gems/elasticity/frames). Be forewarned: Making the calls directly requires that you understand how to structure EMR requests at the Amazon API level and from experience I can tell you there are more fun things you could be doing :) Scroll to the end for more information on the Amazon API.
@@ -113,22 +113,45 @@
jobflow.job_flow_role = nil
jobflow.service_role = nil
jobflow.action_on_failure = 'TERMINATE_JOB_FLOW'
jobflow.keep_job_flow_alive_when_no_steps = false
-jobflow.ami_version = 'latest'
jobflow.log_uri = nil
jobflow.enable_debugging = false # Requires a log_uri to enable
+# >= 4.0.0 release label is now the default
+jobflow.release_label = '4.3.0'
+# < 4.0.0 ... Haven't used this before? just set the release label then.
+jobflow.ami_version = 'latest'
+
jobflow.tags = {name: "app-name", department: 'marketing'}
jobflow.ec2_key_name = nil
jobflow.visible_to_all_users = false
jobflow.placement = 'us-east-1a'
jobflow.region = 'us-east-1'
jobflow.instance_count = 2
jobflow.master_instance_type = 'm1.small'
jobflow.slave_instance_type = 'm1.small'
```
+
+### EMR Applications (optional needs release_label >= 4.0.0)
+With the release of EMR 4.0.0 you can now supply applications which EMR will install for you on boot(rather than a manual bootstrap action. Which you can still use if required). You must set the `release_label` for the jobflow(>=4.0.0)
+
+```ruby
+jobflow.release_label = '4.3.0'
+# the simple way
+jobflow.add_application("Spark") # Pig, Hive, Mahout
+# more verbose
+spark = Elasticity::Application.new({
+ name: 'Spark',
+ arguments: '--webui-port 18080',
+ version: '1.0.1',
+ additional_info: '' # This option is for advanced users only. This is meta information about third-party applications that third-party vendors use for testing purposes.
+})
+jobflow.add_application(spark)
+```
+
+Further reading: http://docs.aws.amazon.com/ElasticMapReduce/latest/ReleaseGuide/emr-configure-apps.html
## 4 - Configure Instance Groups (optional)
Technically this is optional since Elasticity creates MASTER and CORE instance groups for you (one m1.small instance in each). If you'd like your jobs to finish in an appreciable amount of time, you'll want to at least add a few instances to the CORE group :)