README.md in multi_sync-0.0.1 vs README.md in multi_sync-0.0.2

- old
+ new

@@ -1,104 +1,148 @@ # MultiSync -:heavy_exclamation_mark: **a WIP that is functioning but just a warning, not finished yet.** :heavy_exclamation_mark: +:heavy_exclamation_mark: **currently a functioning WIP thats not quite finished yet but its close!** :heavy_exclamation_mark: A flexible synchronisation library for your assets. -At MultiSync's core is [Celluloid](http://celluloid.io) allowing for the synchronisation of assets to be truly parallel. Each target you define creates a pool of resources which allows for parallel execution of uploads and deletes. Meaning that when your synchronising thousands of files, you get alot more :boom: for your :dollar:. +`MultiSync` stands on the shoulders of giants. On one side is [Celluloid](http://celluloid.io) allowing for the synchronisation of assets to be highly parallel. On the other is [Fog::Storage](https://github.com/fog/fog) allowing `MulitSync` to support [various well known storage services](#storage-services). -MultiSync tries to expose its asset synchronisation in a flexible way, allowing for it to be used in `Rails`, `Sinatra (WIP)`, `Rake (WIP)` and `Plain old ruby` as well as extensions for `Middleman (WIP)`, `Nanoc (WIP)` and others. Listed below are examples of how to get setup. +What that means is when your configuring `MultiSync` your creating various pools of workers which then distrubute the work behind synchronising your assets. Meaning that when your site has thousands of files, you get alot more :boom: for your :dollar: in less :alarm_clock:. +`MultiSync` tries to expose its asset synchronisation in a flexible way which should allow you to define how and where your assets live. Where possible though, `MultiSync` will try to provide support for [various well known libraries](#supported-libraries). + +Listed below are examples of how to get setup and started. + ## Installation ```ruby gem 'multi_sync', '~> 0.0.1' ``` ```ruby require 'multi_sync' -MultiSync.configuration do |config| +MultiSync.configure do |config| # config.verbose = false # turn on verbose logging (defaults to false) # config.force = false # force syncing of outdated_files (defaults to false) # config.run_on_build = true # when within a framework which `builds` assets, whether to sync afterwards (defaults to true) # config.sync_outdated_files = true # when an outdated file is found whether to replace it (defaults to true) # config.delete_abandoned_files = true # when an abondoned file is found whether to remove it (defaults to true) # config.upload_missing_files = true # when a missing file is found whether to upload it (defaults to true) # config.target_pool_size = 8 # how many threads you would like to open for each target (defaults to the amount of CPU core's your machine has) - # config.max_sync_attempts = 1 # how many times a file should be retried if there was an error during sync (defaults to 3) + # config.max_sync_attempts = 3 # how many times a file should be retried if there was an error during sync (defaults to 3) end ``` ### Fog credentials support -`MultiSync` supports utilising [Fog Credentials](http://fog.io/about/getting_started.html#credentials). Simply specify either a `FOG_RC` or `.fog` and we'll use it as the base for any credentials used in a target. +`MultiSync` supports utilising [Fog Credentials](http://fog.io/about/getting_started.html#credentials). Simply specify either a `FOG_RC` or `.fog` and we'll use it as the base for any `:credentials` used in a `target`. ## Features / Usage Examples -### AssetSync compatibility +`MultiSync` in its simplist form consists of three objects. `sources`, `resources` and `targets`. A `source` defines how and where a list of files (or `resources`) can be found. A `resource` represents a file from a `source` with additional properties (such as how to compare them). A `target` is destination which `resources` can be synchronised against. -Many people use [AssetSync](https://github.com/rumblelabs/asset_sync) and for `MultiSync`'s first release compatibility with it has been built in. When within a `Rails` environment `MultiSync` will check for `asset_sync.yml` and read in its settings. You should be able to simply require `multi_sync` and try things out. +### Source -#### Unsupported features +A source takes two arguments. The first is a `name` to reference this source by and the second is a `Hash` of configuration detailed below. -- `AssetSync`'s environment variables -- `AssetSync`'s gzip_compression hack +| Key | Type | Default | Description | +| :-- | :--- | :------ | :---------- | +| `type` | `Symbol` | `nil` | The `type` of source this is (`:local`, `:manifest`) | +| `source_dir` | `Pathname`, `String` | `nil` | The location this source should use | +| `resource_options` | `Hash` | `nil` | A hash of options for this source`s resources | +| `targets` | `Symbol`, `Array` | All targets | The target(s) this source should sync against | +| `include` | `String` ([shell glob](http://www.ruby-doc.org/core-2.1.1/Dir.html#method-c-glob)) | `**/*` | A shell globe to use for inclusion | +| `exclude` | `String` ([shell glob](http://www.ruby-doc.org/core-2.1.1/Dir.html#method-c-glob)) | `nil` | A shell globe to use for exclusion | +___ -### Rails +```ruby +# A source named ':build' which is ':local' and will use all files within '../build' +source :build, { + type: :local, + source_dir: '../build' +} +``` +___ -`MultiSync` prefers ruby backed configuration instead of `YAML` so you'll need to create an initializer `/config/initializers/multi_sync` for `MultiSync`. +```ruby +# A source named ':assets' which will use a Sprockets ':manifest' within '../public/assets' +source :assets, { + type: :manifest, + source_dir: '../public/assets' +} +``` +___ ```ruby -MultiSync.prepare do +# A source named ':video_assets' which is `:local' and will use all files +# within '../build' including only 'mp4, mpg, mov' +source :video_assets, { + type: :local, + source_dir: '../build', + include: '*.{mp4,mpg,mov}' +} +``` +___ - source :public, { - type: :manifest, # load files from a Sprockets based manifest - source_dir: '/path_to_your_build_folder', - targets: [:assets] # an array of target names that this source should sync against - } +```ruby +# A source named ':no_images' which is `:local' and will use all files +# within '../build' excluding any 'jpg, gif, png' +source :no_images, { + type: :local, + source_dir: '../build', + exclude: '*.{jpg,gif,png}' +} +``` +___ - target :assets, { - type: :aws, # :aws is the target's type, current options are :aws - target_dir: 'your_aws_bucket', - destination_dir: 'an_optional_directory_inside_your_aws_bucket', - credentials: { - region: 'us-east-1', - aws_access_key_id: 'super_secret', - aws_secret_access_key: 'super_secret' - } - } +```ruby +# A source named ':www' which will use a Sprockets ':manifest' +# within '../public/assets' excluding any 'jpg, gif, png' files +# and only synchronising with a target named `:www` +source :www, { + type: :manifest, + source_dir: '../public/assets', + exclude: '*.{jpg,gif,png}', + targets: :www +} +``` +___ -end +```ruby +# A source named ':image_assets' which will use a Sprockets ':manifest' +# within '../public/assets' including only 'jpg, gif, png' files +# which sets `cache_control` and `expires` headers and +# synchronises with the target `:images` +source :image_assets, { + type: :manifest, + source_dir: '../public/assets', + include: '*.{jpg,gif,png}', + resource_options: { + cache_control: 'public, max-age=31557600', + expires: CGI.rfc1123_date(Time.now + 31557600) + }, + targets: :images +} ``` -`MultiSync.prepare` simply bootstraps `MultiSync` which is then ran during `rake assets:precompile`. By having `multi_sync` included in your `Gemfile`, the rake task `rake assets:sync` will be available. +### Target -### Plain Old Ruby - ```ruby -MultiSync.run do +... +``` - source :build, { - type: :local, # :local is the source's type, current options are :local, :manifest - source_dir: '/path_to_your_build_folder', - targets: [:assets] # an array of target names that this source should sync against - } - target :assets, { - type: :aws, # :aws is the target's type, current options are :aws - target_dir: 'your_aws_bucket', - destination_dir: 'an_optional_directory_inside_your_aws_bucket', - credentials: { - region: 'us-east-1', - aws_access_key_id: 'super_secret', - aws_secret_access_key: 'super_secret' - } - } +## Supported Libraries -end -``` +- [Rails](https://github.com/karlfreeman/multi_sync/wiki/rails) +- Sinatra (WIP) +- Middleman (WIP) +- Jekyll (WIP) +- Nanoc (WIP) +- [POR](https://github.com/karlfreeman/multi_sync/wiki/por) +- Rake (WIP) ## Badges [![Gem Version](http://img.shields.io/gem/v/multi_sync.svg)][gem] [![Build Status](http://img.shields.io/travis/karlfreeman/multi_sync.svg)][travis] @@ -108,11 +152,11 @@ ## Supported Storage Services Behind the scenes we're using [Fog::Storage](http://fog.io/storage) which allows us to support the most popular storage providers - [Amazon S3](http://aws.amazon.com/s3) -- [Rackspace CloudFiles](http://www.rackspace.com/cloud/files) -- [Google Cloud Storage](https://developers.google.com/storage) +- [Rackspace CloudFiles](http://www.rackspace.com/cloud/files) (WIP) +- [Google Cloud Storage](https://developers.google.com/storage) (WIP) ## Supported Ruby Versions This library aims to support and is [tested against][travis] the following Ruby implementations: