# Nightcrawler Swift
[![Gem Version](http://img.shields.io/gem/v/nightcrawler_swift.svg?style=flat-square)](http://rubygems.org/gems/nightcrawler_swift)
[![Build Status](http://img.shields.io/travis/tulios/nightcrawler_swift.svg?style=flat-square)](https://travis-ci.org/tulios/nightcrawler_swift)
[![Code Climate](https://img.shields.io/codeclimate/github/tulios/nightcrawler_swift.svg?style=flat-square)](https://codeclimate.com/github/tulios/nightcrawler_swift)


Like the X-Men nightcrawler this gem teleports your assets to a OpenStack Swift bucket/container. It was designed to sync your assets with OpenStack Swift and allow some operations with your buckets/containers.

## Compatibility

nightcrawler_swift version 1.x is compatible with [Identity API v3](https://developer.openstack.org/api-ref/identity/v3/index.html). To use [API v2](https://developer.openstack.org/api-ref/identity/v2/index.html), you should install version 0.x.

## Installation

Add this line to your application's Gemfile:

```ruby
gem 'nightcrawler_swift'
```

Or, if you want to use Identity API v2:

```ruby
gem 'nightcrawler_swift', '< 1.0'
```

And then execute:

```sh
$ bundle
```

Or install it manually:

```sh
$ gem install nightcrawler_swift  # Install latest version
```

Or set a specific version:

```sh
$ gem install nightcrawler_swift -v 0.11.1
```

## Usage

* [With Rails](https://github.com/tulios/nightcrawler_swift#with-rails)
* [Programatically](https://github.com/tulios/nightcrawler_swift#programatically)
* [Command Line](https://github.com/tulios/nightcrawler_swift#command-line)

### With Rails
#### 1) Configure your swift credentials and options

_In config/application.rb_ or _config/environments/*.rb_

```ruby
config.nightcrawler_swift.bucket = "rogue"
config.nightcrawler_swift.tenant_name = "nightcrawler"
config.nightcrawler_swift.username = "my_username1"
config.nightcrawler_swift.password = "my_password1"
config.nightcrawler_swift.auth_url = "https://auth.url.com:123/v2.0/tokens"
```

The __password__ option can be left blank and be defined by the env variable __NSWIFT_PASSWORD__. The env variable will take precedence if the option was set.

__Optional configurations:__

```ruby
config.nightcrawler_swift.max_age = 3600 # default: nil
config.nightcrawler_swift.timeout = 10 # in seconds, default: nil

# default: uses the admin_url returned by authentication
config.nightcrawler_swift.admin_url = "https://api.host.com/v1/AUTH_1234"

# default: uses the public_url returned by authentication
config.nightcrawler_swift.public_url = "http://asset.host.com/v1/AUTH_1234"

# default: 5, to disable set it to false
config.nightcrawler_swift.retries = 3

# in seconds, default: 30
config.nightcrawler_swift.max_retry_time = 64

# default: false. You could use OpenSSL::SSL::VERIFY_PEER
config.nightcrawler_swift.verify_ssl = true

# default: nil
config.nightcrawler_swift.ssl_client_cert =
  OpenSSL::X509::Certificate.new(File.read("cert.pem"))

# default: nil
config.nightcrawler_swift.ssl_client_key =
  OpenSSL::PKey::RSA.new(File.read("key.pem"), "passphrase, if any")

# default: nil
config.nightcrawler_swift.ssl_ca_file = "ca_certificate.pem"

#default: nil
config.nightcrawler_swift.ssl_version = "SSLv23"

#default: {}
config.nightcrawler_swift.custom_headers = {custom_header: 'custom_value'}
```

By default it will use ```Rails.logger``` as logger, to change that use a different logger in configurations, like:

```ruby
config.nightcrawler_swift.logger = Logger.new(STDOUT)
```

##### further explanation of configurations

> max_age

Defines the *Cache-Control:max-age=<value>* header.

> expires

Defines the *Expires* header.

> content_encoding

Defines the *Content-Encoding:<vaule>* header

> retries

The number of times to retry the request before failing. To disable this feature set it to __false__.

> max_retry_time

Maximum delay in seconds between each retry. The delay will start with 1s and will double for each retry until this value.

> custom_headers

Optional hash with header fields to be uploaded with the object.

#### 2) Profit!

```sh
$ rake nightcrawler_swift:rails:asset_sync
```

It will invoke ```rake assets:precompile``` and will copy your public directory to swift bucket/container. To sync the public directory without the asset precompilation use the task: ```nightcrawler_swift:rails:sync```

### Programatically

#### 1) Configure your swift credentials and options

```ruby
NightcrawlerSwift.configure({
  bucket: "rogue",
  tenant_name: "nightcrawler",
  username: "my_username1",
  password: "my_password1",
  auth_url: "https://auth.url.com:123/v2.0/tokens"
})
```

The __password__ option can be left blank and be defined by the env variable __NSWIFT_PASSWORD__. The env variable will take precedence if the option was set.


__Optional configurations:__

```ruby
max_age: 3600,
timeout: 10, # in seconds

# default: uses the admin_url returned by authentication
admin_url: "https://api.host.com/v1/AUTH_1234",

# default: uses the public_url returned by authentication
public_url: "http://asset.host.com/v1/AUTH_1234",

# default: 5, to disable set it to false
retries: 3,

# in seconds, default: 30
max_retry_time: 64

# default: false. You could use OpenSSL::SSL::VERIFY_PEER
verify_ssl: true,

# default: nil
ssl_client_cert:
  OpenSSL::X509::Certificate.new(File.read("cert.pem")),

# default: nil
ssl_client_key:
  OpenSSL::PKey::RSA.new(File.read("key.pem"), "passphrase, if any"),

# default: nil
ssl_ca_file: "ca_certificate.pem"

# default: {}
custom_headers: {custom_header: 'custom_value'}
```

By default it will use ```Logger.new(STDOUT)``` as logger, to change that use:

```ruby
NightcrawlerSwift.logger = Logger.new(StringIO.new)
```

#### 2) Use method sync with the desired directory

```ruby
NightcrawlerSwift.sync File.expand_path("./my-dir")
```

### Command Line

The NightcrawlerSwift shell command (CLI) allows you to interact with your buckets/containers easily, it has the same commands of the gem. To see the help, use the cli without arguments or use the _-h_/_--help_ switch.

```sh
$ nswift # or nswift -h
```

```nswift``` will use the configurations stored at the file __.nswiftrc__ located at your home directory. If you try to use any command without the file, it will create a sample configuration for you, but you can create your own.

The configuration is a __json__ file, named __.nswiftrc__. You can include any configuration available to the gem (see the other usages example to know each option available). Follow the format:

```json
{
  "bucket": "<bucket/container name>",
  "tenant_name": "<tenant name>",
  "username": "<username>",
  "password": "<password>",
  "auth_url": "<auth url, ex: https://auth.url.com:123/v2.0/tokens>"
}
```

The following commands are available through the cli:

```sh
$ nswift list
```
```sh
$ nswift upload <real_path> <swift_path> # nswift upload robots.txt assets/robots.txt
```

Upload also supports a custom max-age, to override the value defined in ".nswiftrc", example:

```sh
$ nswift upload <real_path> <swift_path> --max-age VALUE # nswift upload readme assets/readme.md --max-age 300
```

```sh
$ nswift download <swift_path> # nswift download assets/robots.txt > my-robots.txt
```
```sh
$ nswift delete <swift_path> # nswift delete assets/robots.txt
```
```sh
$ nswift url-for <swift_path> # nswift url-for assets/robots.txt
```
```sh
$ nswift metadata <swift_path>
# {
#   "date": "Tue, 31 Mar 2015 02:10:40 GMT",
#   "content_type": "text/css",
#   "content_length": "50013",
#   "last_modified": "Wed, 15 Oct 2014 13:38:56 GMT",
#   "etag": "e13839c545f32be23b942a41f3ea7724",
#   "x_timestamp": "1413380335.38118",
#   "cache_control": "public, max-age=604800",
#   "x_trans_id": "tx824d632ef9e54685a7563-005519f838"
# }
```

For any commands you could provide a different configuration file through the _-c_/_--config_ switch, as:

```sh
$ nswift list -c /dir/my-nswift-rc
```

and a different bucket/container name through the _-b_/_--bucket_ switch, as:

```sh
$ nwift list -b rogue
```

## Commands

NightcrawlerSwift has some useful built-in commands. All commands require the configuration and will __automatically__ connect/reconnect to keystone when necessary.

### Upload

```ruby
upload = NightcrawlerSwift::Upload.new
upload.execute "my_file_path.txt", File.open("../my_file_fullpath.txt", "r")
# true / false
```

_This upload command was not designed to send very large files_.

It will accept a custom max-age, overriding the configured value through ```NightcrawlerSwift.configure```, example:

```ruby
upload = NightcrawlerSwift::Upload.new
upload.execute "readme", File.open("readme.md", "r"), max_age: 300
# true / false
```

### Download

```ruby
download = NightcrawlerSwift::Download.new
download.execute "my_file_path.txt"
# File content
```

_This download command was not designed to retrieve very large files_

### List

```ruby
list = NightcrawlerSwift::List.new
list.execute
# [{"hash": "...", "name": "my_file_path.txt"}, {}, {}, ...]
```

This command supports the following parameters: __limit, marker, end_marker, prefix, format, delimiter, path__

Example:

```ruby
list = NightcrawlerSwift::List.new
list.execute prefix: '/some/path'
# [{"hash": "...", "name": "/some/path/with/my_file_path.txt"}, {}, {}, ...]
```

### Metadata

```ruby
metadata = NightcrawlerSwift::Metadata.new
metadata.execute "my_file.css"
# {
#   :date=>"Tue, 31 Mar 2015 01:46:10 GMT",
#   :content_type=>"text/css",
#   :content_length=>"50013",
#   :last_modified=>"Wed, 15 Oct 2014 13:38:56 GMT",
#   :etag=>"e13839c545f32be23b942a41f3ea7724",
#   :x_timestamp=>"1413380335.38118",
#   :cache_control=>"public, max-age=604800",
#   :x_trans_id=>"tx92ca1becd1a74fa3ae1f2-005519fc62"
# }

# Consulting the bucket/container metadata
metadata.execute
# {
#   :date=>"Tue, 31 Mar 2015 01:41:25 GMT",
#   :content_type=>"application/xml; charset=utf-8",
#   :content_length=>"98171",
#   :x_container_object_count=>"374",
#   :x_storage_policy=>"default-3x",
#   :x_container_read=>".r:*",
#   :x_container_bytes_used=>"7171468",
#   :x_timestamp=>"1409750420.36261",
#   :x_trans_id=>"tx17d8c66ab746436b8b7b6-005519fb40"
# }
```

### Delete

```ruby
delete = NightcrawlerSwift::Delete.new
delete.execute "my_file_path.txt"
# true / false
```

### Sync

```ruby
sync = NightcrawlerSwift::Sync.new
sync.execute "/dir/to/synchronize"
```

## Connection

To manually establish the connection with keystone, use:

```ruby
NightcrawlerSwift.connection.connect!
```

To check if the connection is still valid, use:

```ruby
NightcrawlerSwift.connection.connected?
```

To reconnect just use ```NightcrawlerSwift.connection.connect!``` again.

## Options

After configure the NightcrawlerSwift you can access your configurations through the __options__ method, like:

```ruby
NightcrawlerSwift.options
```

The only difference is that you will access each configuration as a method instead of a hash style, like:

```ruby
NightcrawlerSwift.configure tenant_name: "rogue"

# Can be used as:
NightcrawlerSwift.options.tenant_name # "rogue"
```

## Contributors

Check it out!

https://github.com/tulios/nightcrawler_swift/graphs/contributors

## License

See [LICENSE](https://github.com/tulios/nightcrawler_swift/blob/master/LICENSE.txt) for more details.

## Contributing

1. Fork it ( https://github.com/tulios/nightcrawler_swift/fork )
2. Create your feature branch (`git checkout -b my-new-feature`)
3. Commit your changes (`git commit -am 'Add some feature'`)
4. Push to the branch (`git push origin my-new-feature`)
5. Create a new Pull Request