Contents

#!/usr/bin/env ruby

require 'wukong-load'
settings = Wukong::Load::Configuration
settings.use(:commandline)

settings.usage = "usage: #{File.basename($0)} DATA_STORE [ --param=value | -p value | --param | -p]"
settings.description = <<-EOF
wu-load is a tool for loading data from Wukong into data stores.  It
supports multiple, pluggable data stores, including:

Supported data stores:

   elasticsearch
   hbase (planned)
   mongob (planned)
   mysql (planned)

Get specific help for a data store with

  $ wu-load store_name --help

Elasticsearch Usage:

Pass newline-separated, JSON-formatted records over STDIN:

$ cat data.json | wu-load elasticsearch

By default, wu-load attempts to write each input record to a local
Elasticsearch database.  Records will be routed to a default
Elasticsearch index and type.  Records with an '_id' field will be
considered updates.  The rest will be creates.  You can override these
options:

$ cat data.json | wu-load elasticsearch --host=10.123.123.123 --index=my_app --es_type=my_obj --id_field="doc_id"

Params:
   --host=String            Elasticsearch host, without HTTP prefix [Default: localhost]
   --port=Integer           Port on Elasticsearch host [Default: 9200]
   --index=String           Default Elasticsearch index for records [Default: wukong]
   --es_type=String         Default Elasticsearch type  for records [Default: streaming_record]
   --index_field=String     Field in each record naming desired Elasticsearch index
   --es_type_field=String   Field in each record naming desired Elasticsearch type
   --id_field=String        Field in each record naming providing ID of existing Elasticsearch record to update
EOF

require 'wukong/boot' ; Wukong.boot!(settings)

require 'wukong-load/runner'
Wukong::Load::Runner.run(settings)

Version data entries

1 entries across 1 versions & 1 rubygems

Version	Path
wukong-load-0.0.2	bin/wu-load