README.md in saviour-0.5.10 vs README.md in saviour-0.5.11
- old
+ new
@@ -2,571 +2,1164 @@
[](https://codeclimate.com/github/rogercampos/saviour)
[](https://codeclimate.com/github/rogercampos/saviour/coverage)
# Saviour
-This is a small library that handles file uploads and nothing more. It integrates with ActiveRecord and manages file
-storage following the active record instance lifecycle.
+Saviour is a tool to help you manage files attached to Active Record models. It tries to be minimal about the
+use cases it covers, but with a deep and complete coverage on the ones it does. For example, it offers
+no support for image manipulation, but it does implement dirty tracking and transactional-aware behavior.
+It also tries to have a flexible design, so that additional features can be added by the user on top of it.
+You can see an example of such typical features on the [FAQ section at the end of this document](#faq).
+
+## Motivation
+
+This project started in 2015 as an attempt to replace Carrierwave. Since then other solutions have appeared
+to solve the same problem, like [shrine](https://github.com/shrinerb/shrine), [refile](https://github.com/refile/refile)
+and even more recently rails own solution [activestorage](https://github.com/rails/rails/tree/master/activestorage).
+
+The main difference between those solutions and Saviour is about the broadness and scope of the problem
+that wants to be solved.
+
+They offer a complete out-of-the-box solution that covers many different needs:
+image management, caching of files for seamless integration with html forms, direct uploads to s3, metadata
+extraction, background jobs integration or support for different ORMs are some of the features you can find on
+those libraries.
+
+If you need those functionalities and they suit your needs, they can be perfect solutions for you.
+
+The counterpart, however, is that they have more dependencies and, as they cover a broader spectrum of
+use cases, they tend to impose more conventions that are expected to be followed as is. If you don't want,
+or can't follow some of those conventions then you're out of luck.
+
+Saviour provides a battle-tested infrastructure for storing files following an AR model
+life-cycle which can be easily extended to suit your custom needs.
+
+
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'saviour'
+```
+
+And then execute:
+
+ $ bundle
+
+
<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
-
-- [Intro](#intro)
-- [Basic usage example](#basic-usage-example)
-- [File API](#file-api)
-- [Storage abstraction](#storage-abstraction)
- - [public_url](#public_url)
- - [LocalStorage](#localstorage)
- - [S3Storage](#s3storage)
-- [Source abstraction](#source-abstraction)
- - [StringSource](#stringsource)
- - [UrlSource](#urlsource)
-- [Uploader classes and Processors](#uploader-classes-and-processors)
- - [store_dir](#store_dir)
- - [Accessing model and attached_as](#accessing-model-and-attached_as)
- - [Processors](#processors)
-- [Versions](#versions)
-- [Validations](#validations)
-- [Active Record Lifecycle integration](#active-record-lifecycle-integration)
+- [Quick start](#quick-start)
+ - [General Usage](#general-usage)
+ - [Api on attachment](#api-on-attachment)
+ - [Additional api on the model](#additional-api-on-the-model)
+ - [Storages](#storages)
+ - [Local Storage](#local-storage)
+ - [S3 Storage](#s3-storage)
+ - [Uploader classes](#uploader-classes)
+ - [store_dir](#store_dir)
+ - [Processors](#processors)
+ - [halt_process](#halt_process)
+ - [Versions](#versions)
+ - [Transactional behavior](#transactional-behavior)
+ - [Concurrency](#concurrency)
+ - [stash](#stash)
+ - [Dirty tracking](#dirty-tracking)
+ - [AR Validations](#ar-validations)
+ - [Introspection](#introspection)
+- [Extras & Advance usage](#extras--advance-usage)
+ - [Skip processors](#skip-processors)
+ - [Testing](#testing)
+ - [Sources: url and string](#sources-url-and-string)
+ - [Custom Storages](#custom-storages)
+ - [Bypassing Saviour](#bypassing-saviour)
+ - [Bypass example: Nested Cloning](#bypass-example-nested-cloning)
- [FAQ](#faq)
- - [Digested filename](#digested-filename)
- - [Getting metadata from the file](#getting-metadata-from-the-file)
+ - [how to reuse code in your app, attachment with defaults](#how-to-reuse-code-in-your-app-attachment-with-defaults)
+ - [How to manage file removal from forms](#how-to-manage-file-removal-from-forms)
+ - [How to extract metadata from files](#how-to-extract-metadata-from-files)
+ - [How to process files in background / delayed](#how-to-process-files-in-background--delayed)
- [How to recreate versions](#how-to-recreate-versions)
- - [Caching across redisplays in normal forms](#caching-across-redisplays-in-normal-forms)
- - [Introspection (Class.attached_files)](#introspection-classattached_files)
- - [Processing in background](#processing-in-background)
+ - [How to digest the filename](#how-to-digest-the-filename)
+- [License](#license)
<!-- END doctoc generated TOC please keep comment here to allow auto update -->
-## Intro
+## Quick start
-The goal of this library is to be as minimal as possible, including as less features and code the better. This library's
-responsibility is to handle the storage of a file related to an ActiveRecord object, persisting the file on save and
-deleting it on destroy. Therefore, there is no code included to handle images, integration with rails views or any
-other related feature. There is however a FAQ section later on in this README that can help you implement those things
-using Saviour and your own code.
+First, you'll need to configure Saviour to indicate what type of storage you'll want to use. For example,
+to use local storage:
+```ruby
+# config/initializers/saviour.rb
-## Basic usage example
+Saviour::Config.storage = Saviour::LocalStorage.new(
+ local_prefix: Rails.root.join('public/system/uploads/'),
+ public_url_prefix: "https://mywebsite.com/system/"
+)
+```
-This library is inspired api-wise by carrierwave, sharing the same way of declaring "attachments" (file storages related
-to an ActiveRecord object) and processings. See the following example of a model including a file:
+A local storage will persist the files on the server running the ruby code and will require settings to
+indicate precisely where to store those files locally and how to build a public url to them. Those settings
+depend on your server and deployment configurations. Saviour ships with local storage and Amazon's S3 storage
+capabilities, see the section on [Storages](#storages) for more details.
-```
-class Post < ActiveRecord::Base
- include Saviour::Model
+Saviour will also require a text column for each attachment in an ActiveRecord model. This column will be used to
+persist a file's "path" across the storage. For example:
- # The posts table must have an `image` string column.
- attach_file :image, PostImageUploader
+```ruby
+create_table "users" do |t|
+ # other columns...
+ t.text "avatar"
end
+```
-class PostImageUploader < Saviour::BaseUploader
- store_dir { "/default/path/#{model.id}/#{attached_as}" }
+Then include the mixin `Saviour::Model` in your AR model and declare the attachment:
- process :resize, width: 500, height: 500
+```ruby
+class User < ApplicationRecord
+ include Saviour::Model
- version(:thumb) do
- process :resize, width: 100, height: 100
+ attach_file(:avatar) do
+ store_dir { "uploads/avatars/#{model.id}/" }
end
+end
+```
- def resize(contents, filename, opts)
- width = opts[:width]
- height = opts[:height]
+Declaring a `store_dir` is mandatory and indicates at what base path the assigned files must be stored. More
+on this later at the [Uploaders section](#uploader-classes).
- # modify contents in memory here
- contents = user_implementation_of_resize(contents, width, height)
+Now you can use it:
- [contents, filename]
- end
-end
+```ruby
+user = User.create! avatar: File.open("/path/to/cowboy.jpg")
+user.avatar.read # => binary contents
+
+# Url generation depends on how the storage is configured
+user.avatar.url # => "https://mywebsite.com/system/uploads/avatars/1/cowboy.jpg"
+
+# Using local storage, the persisted column will have the path to the file
+user[:avatar] # => "uploads/avatars/1/cowboy.jpg"
```
-In this example we have posts that have an image. That image will be stored in a path like `/default/path/<id>/image`
-and also a resize operation will be performed before persisting the file.
-There's one version declared with the name `thumb` that will be created by resizing the file to 100x100. The version
-filename will be by default `<original_filename>_thumb` but it can be changed if you want.
+### General Usage
-Filenames (both for the original image and for the versions) can be changed in a processor just by returning a different
-second argument.
+You can assign to an attachment any object that responds to `read`. This includes `File`, `StringIO` and many others.
-Here the resize manipulation is done in-memory, but there're also a way to handle manipulations done at the file level if
-you need to use external binaries like imagemagick, image optimization tools (pngquant, jpegotim, etc...) or others.
+The filename given to the file will be obtained by following this process:
+- First, trying to call `original_filename` on the given object.
+- Second, trying to call `filename` on the given object.
+- Finally, if that object responds to `path`, it will be extracted as the basename of that path.
-## File API
+If none of that works, a random filename will be assigned.
-`Saviour::File` is the type of object you'll get when accessing the attribute over which a file is attached in the
-ActiveRecord object. The public api you can use on those objects is:
+The actual storing of the file and any possible related processing (more on this [later](#processors)) will
+happen on after save, not on assignation. You can assign and re-assign different values to an attachment at no
+cost.
-- assign
-- exists?
-- read
-- write
-- delete
-- public_url
-- url
-- changed?
-- filename
-- with_copy
-- blank?
-Use `assign` to assign a file to be stored. You can use any object that responds to `read`. See below the section
-about Sources abstraction for further info.
+#### Api on attachment
-`exists?`, `read`, `write`, `delete` and `public_url` are delegated to the storage, with the exception of `write` that
-is channeled with the uploader first to handle processings. `url` is just an alias for `public_url`.
+Given the previous example of a User with an avatar attachment, the following methods are available to you on the attachment object:
-`changed?` indicates if the file has changed, in memory, regarding it's initial value. It's equivalent to the `changed?`
-method that ActiveRecord implements on database columns.
+- `user.avatar.present?` && `.blank?`: Indicates if the attachment has an associated file or not, even if it has not been persisted yet. This methods allow you for a transparent use of rails `validates_presence_of :avatar`, as the object responds to `blank?`.
+- `user.avatar.persisted?`: Indicates if the attachment has an associated file and this file is persisted. Is false after assignation and before save.
+- `user.avatar?`: Same as `user.avatar.present?`
+- `user.avatar.exists?`: If the attachment is `persisted?`, it checks with the storage to verify the existence of the associated path. Use it to check for situations where the database has a persisted path but the storage may not have the file, due to any other reasons (direct manipulation by other means).
+- `user.avatar.with_copy {|f| ... }`: Utility method that fetches the file and gives it to you in the form of a `Tempfile`. Will forward the return value of your block. The tempfile will be cleaned up on block termination.
+- `user.avatar.read`: Returns binary raw contents of the stored file.
+- `user.avatar.url`: Returns the url to the stored file, based on the storage configurations.
+- `user.avatar.reload`: If the contents of the storage were directly manipulated, you can use this method to force a reload of the attachment state from the storage.
+- `user.avatar.filename`: Returns the filename of the stored file.
+- `user.avatar.persisted_path`: If persisted, returns the path of the file as stored in the storage, otherwise nil. It's the same as the db column value.
+- `user.avatar.changed?`: Returns true/false if the attachment has been assigned but not yet saved.
+
+Usage example:
-`filename` is the filename of the currently stored file. Only works for files that have been already stored, not assigned.
+```ruby
+user = User.new
-`blank?` indicates if the file is present either in the persistence layer or in memory. It provides api-compatibility with
-default rails validations like `validates_presence_of`.
+user.avatar? # => false
+user.avatar.present? # => false
+user.avatar.blank? # => true
-`with_copy` is a helper method that will read the persisted file, create a copy using a `Tempfile` and call the block
-passed to the method with that Tempfile. Will clean afterwards.
+user.avatar.read # => nil, same for #url, #filename, #persisted_path
-As mentioned before, you can access a `File` object via the name of the attached_as, from the previous example you could do:
+user.avatar = File.open("image.jpg")
+user.avatar.changed? # => true
+user.avatar? # => true, same as #present?
+user.avatar.persisted? # => false
+
+user.avatar.url # => nil, not yet persisted
+user.avatar.exists? # => false, not yet persisted
+user.avatar.filename # => "image.jpg"
+user.avatar.read # => nil, not yet persisted
+
+user.avatar.with_copy # => nil, not yet persisted
+
+user.save!
+
+user.avatar.changed? # => false
+user.avatar? # => true
+user.avatar.exists? # => true
+user.avatar.persisted? # => true
+
+user.avatar.read # => bytecontents
+user.avatar.url # => "https://somedomain.com/path/image.jpg"
+user.avatar.with_copy # => yields a tempfile with the image
+user.avatar.read # => bytecontents
```
-post = Post.find(123)
-post.image # => <Saviour::File>
+
+
+#### Additional api on the model
+
+When you declare an attachment in an AR model, the model is extended with:
+
+- `#dup`: The `dup` method over the AR instance will also take care of dupping any possible attachment with associated files if any. If the new instance returned by dup is saved, the attachments will be saved as well normally, generating a copy of the files present on the original instance.
+
+- `#remove_<attached_as>!`: This new method will be added for each attachment. For example, `user.remove_avatar!`. Use this method to remove the associated file.
+
+Usage example:
+
+```ruby
+user = User.create! avatar: File.open("image.jpg")
+
+user.avatar.url # => "https://somedomain.com/uploads_path/users/1/avatar/image.jpg"
+
+new_user = user.dup
+new_user.save!
+
+new_user.avatar.url # => "https://somedomain.com/uploads_path/users/2/avatar/image.jpg"
+
+new_user.remove_avatar!
+new_user.avatar? # => false
```
-You can also get the `File` instance of version by using an argument matching the version name:
+
+### Storages
+
+Storages are the Saviour's components responsible for file persistence. Local storage and Amazon's S3 storage
+are available by default, but more can be built, as they are designed as independent components and any class
+that follows the expected public api can be used as one. More on this on the [Custom storage section](custom-storages).
+
+We'll review now how to use the two provided storages.
+
+#### Local Storage
+
+You can use this storage to store files in the local machine running the ruby code. Example:
+
+```ruby
+# config/initializers/saviour.rb
+
+Saviour::Config.storage = Saviour::LocalStorage.new(
+ local_prefix: Rails.root.join('public/system/uploads/'),
+ public_url_prefix: "http://mydomain.com/uploads"
+)
```
-post = Post.find(123)
-post.image # => <Saviour::File>
-post.image(:thumb) # => <Saviour::File>
+
+The `local_prefix` is the base prefix under which the storage will store files in the
+machine. You need to configure this accordingly to your use case and deployment strategies, for example, for rails
+and capistrano with default settings you'll have to store the files under `Rails.root.join("public/system")`,
+as this is by default the shared directory between deployments.
+
+The `public_url_prefix` is the base prefix to build the public endpoint from which you'll serve the assets.
+Same as before, you'll need to configure this accordingly to your deployment specifics.
+
+You can also assign a Proc instead of a String to dynamically calculate the value, useful when you have multiple
+asset hosts:
+
+`public_url_prefix: -> { https://media-#{rand(4)}.mywebsite.com/system/uploads/" }`
+
+This storage will take care of removing folders after they become empty.
+
+The optional extra argument `permissions` will allow you to set what permissions the files should have locally.
+This value defaults to '0644' and can be changed when creating the storage instance:
+
+```ruby
+Saviour::Config.storage = Saviour::LocalStorage.new(
+ local_prefix: Rails.root.join('public/system/uploads/'),
+ public_url_prefix: "http://mydomain.com/uploads",
+ permissions: '0600'
+)
```
-Finally, a couple of convenient methods are also added to the ActiveRecord object that just delegate to the `File` object:
+#### S3 Storage
+This storage will store files on Amazon S3, using the `aws-sdk-s3` gem. Example:
+
+```ruby
+Saviour::Config.storage = Saviour::S3Storage.new(
+ bucket: "my-bucket-name",
+ aws_access_key_id: "stub",
+ aws_secret_access_key: "stub",
+ region: "my-region",
+ public_url_prefix: "https://s3-eu-west-1.amazonaws.com/my-bucket/"
+)
```
-post = Post.find(123)
-post.image = File.open("/my/image.jpg") # This is equivalent to post.image.assign(File.open(...))
-post.image_changed? # This is equivalent to post.image.changed?
+
+The first 4 options (`bucket`, `aws_access_key_id`, `aws_secret_access_key` and `region`) are required for the
+connection and usage of your s3 bucket.
+
+The `public_url_prefix` is the base prefix to build the public endpoint from which the files are available.
+Normally you'll set it as in the example provided, or you can also change it accordingly to any CDN you may be
+using.
+
+You can also assign a Proc instead of a String to dynamically calculate the value, which is useful when you have multiple
+asset hosts:
+
+`public_url_prefix: -> { https://media-#{rand(4)}.mywebsite.com/system/uploads/" }`
+
+The optional argument `create_options` can be given to establishing extra parameters to use when creating files. For
+example you might want to set up a large cache control value so that the files become cacheable:
+
+```ruby
+ create_options: {
+ cache_control: 'max-age=31536000' # 1 year
+ }
```
-## Storage abstraction
+Those options will be forwarded directly to aws-sdk, you can see the complete reference here:
-Storages are classes responsible for handling the persistence layer with the underlying persistence provider, whatever
-that is. Storages are considered public API and anyone can write a new one. Included in the Library there are two of them,
-LocalStorage and S3Storage. To be an Storage, a class must implement the following api:
+https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Client.html#put_object-instance_method
+Currently, there's no support for different create options on a per-file basis. All stored files will be created
+using the same options. If you want a public access on those files, you can make them public with a general
+rule at the bucket level or using the `acl` create option:
+
+```ruby
+ create_options: {
+ acl: 'public-read'
+ }
```
-def write(contents, path)
+
+NOTE: Be aware that S3 has a limit of 1024 bytes for the keys (paths) used. Trying to store a file with a
+larger path will result in a `Saviour::KeyTooLarge` exception.
+
+
+### Uploader classes
+
+Uploader classes are responsible to make changes to an attachment byte contents or filename, as well as indicating
+what base path that file should have.
+
+An uploader class can be provided explicitly, for example:
+
+```ruby
+# app/uploaders/post_image_uploader.rb
+class PostImageUploader < Saviour::BaseUploader
+ store_dir { "uploads/posts/images/#{model.id}/" }
end
-def read(path)
+# app/models/post.rb
+class Post < ApplicationRecord
+ include Saviour::Model
+
+ attach_file :image, PostImageUploader
end
+```
-def exists?(path)
+Or you can also provide a `&block` to the `attach_file` method to declare the uploader class implicitly. This
+syntax is usually more convenient if you don't have a lot of code in your uploaders:
+
+```ruby
+class Post < ApplicationRecord
+ include Saviour::Model
+
+ attach_file :image do
+ store_dir { "uploads/posts/images/#{model.id}/" }
+ end
end
+```
-def delete(path)
+
+#### store_dir
+
+Declaring a `store_dir` is mandatory for each uploader class. It can be provided directly as a block or as a symbol,
+in which case it has to match with a method you define on the uploader class.
+
+Its returning value must be a string representing the base path under which the files will be stored.
+
+At runtime the model is available as `model`, and the name of the attachment as `attached_as`. For example:
+
+```ruby
+class PostImageUploader < Saviour::BaseUploader
+ store_dir { "uploads/posts/images/#{model.id}/" }
+
+ # or
+ store_dir { "uploads/posts/#{model.id}/#{attached_as}" }
+
+ # or more generic
+ store_dir { "uploads/#{model.class.name.parameterize}/#{model.id}/#{attached_as}" }
+
+ # or with a method
+ store_dir :calculate_dir
+
+ def calculate_dir
+ "uploads/posts/images/#{model.id}/"
+ end
end
```
-The convention here is that a file consist of a raw content and a path representing its location within the underlying
-persistence layer.
+Since attachment processing and storing happens on after save, at the time `store_dir` is called the model
+has already been saved, so the database `id` is available.
-You must configure Saviour by providing the storage to use:
+The user is expected to configure such store_dirs appropriately so that path collisions cannot happen across
+the whole application. To that end, the use of `model.id` and `attached_as` as part of
+the store dir is a common approach to ensure there will be no collisions. Other options could involve
+random token generation.
+
+#### Processors
+
+Processors are methods (or lambdas) that receive the contents of the file being saved and its filename,
+and in turn return file contents and filename. You can use them to change both values, for example:
+
+```ruby
+class PostImageUploader < Saviour::BaseUploader
+ store_dir { "uploads/posts/#{model.id}/#{attached_as}" }
+
+ process do |contents, filename|
+ new_filename = "#{Digest::MD5.hexdigest(contents)}-#{filename}"
+ new_contents = Zlib::Deflate.deflate(contents)
+
+ [new_contents, new_filename]
+ end
+end
```
-Saviour::Config.storage = MyStorageImplementation.new
+
+Here we're compressing the contents with ruby's zlib and adding a checksum to the filename for caching purposes. The returning
+value must be always an array of two values, a pair of contents/filename.
+
+If you want to reuse processors and make them more generic with variables, you can also define them as methods
+and share them via a ruby module or via inheritance. In this form, you can pass arbitrary arguments.
+
+```ruby
+module ProcessorsHelpers
+ def resize(contents, filename, width:, height:)
+ new_contents = SomeImageManipulationImplementation.new(contents).resize_to(width, height)
+
+ [new_contents, filename]
+ end
+end
+
+class PostImageUploader < Saviour::BaseUploader
+ include ProcessorsHelpers
+ store_dir { "uploads/posts/#{model.id}/#{attached_as}" }
+
+ process :resize, width: 100, height: 100
+end
```
-The provided storage object is considered a global configuration state that will be used by Saviour for all mounters.
-However, this configuration is thread-safe and can be changed at runtime, allowing you in practice to work with different
-storages by swapping them depending on your use case.
+You may declare as many processors as you want in an uploader class, they will be executed in the same order as
+you define them and they will be chained: the output of the first processor will be the input of the second one, etc.
+Inside a processor you also have access to the following variables:
-### public_url
+- `model`: The model owner of the file being saved.
+- `attached_as`: The name of the attachment being processed.
+- `store_dir`: The computed value of the store dir this file will have.
-Storages can optionally also implement this method, in order to provide a public URL to the stored file without going
-through the application code.
+When you use `process` to declare processors as seen before, you're given the raw byte contents that were originally
+assigned to the attachment. This may be convenient if you have a use case when you generate those contents yourself
+or want to manipulate them directly with ruby, but that's normally not the case. Usually, you assign files to
+the attachments and modify them via third party binaries (like imagemagick). In that scenario, in order to reduce
+memory usage, you can use instead `process_with_file`.
-For example, if you're storing files in a machine with a webserver, you may want this method to convert from a local
-path to an external URL, adding the domain and protocol parts. As an ilustrative example:
+This is essentially the same but instead of raw byte contents you're given a `Tempfile` instance:
+```ruby
+class PostImageUploader < Saviour::BaseUploader
+ store_dir { "uploads/posts/#{model.id}/#{attached_as}" }
+
+ process_with_file do |file, filename|
+ `convert -thumbnail 100x100^ #{Shellwords.escape(file.path)}`
+
+ [file, filename]
+ end
+end
```
-def public_url(path)
- "http://mydomain.com/files/#{path}"
+
+*Note that when escaping to the shell you need to check for safety in case there's an injection in the filename.*
+
+You can modify directly the contents of the given file in the filesystem, or you could also delete the given file and
+return a new one. If you return a different file instance, you're expected to clean up the one that was given to you.
+
+You can mix `process` with `process_with_file` but you should try to avoid it, as it will be a performance penalty
+having to convert between formats.
+
+Also, even if there's just one `process`, the whole contents of the file will be loaded into memory. Avoid that usage
+if you're conservative about memory usage or take care of restricting the allowed file size you can work with on
+any file upload you accept across your application.
+
+
+##### halt_process
+
+`halt_process` is a method you can call from inside a processor in order to abort the processing and storing
+of the current file. You can use this to conditionally store a file or not based on runtime decisions.
+
+For example, you may be storing media files that can be audio, video or images, and you want to generate a
+thumbnail for videos and images but not for audio files.
+
+```ruby
+class ThumbImageUploader < Saviour::BaseUploader
+ store_dir { "uploads/thumbs/#{model.id}/#{attached_as}" }
+
+ process_with_file do |file, filename|
+ halt_process unless can_generate_thumb?(file)
+ `convert -thumbnail 100x100^ #{Shellwords.escape(file.path)}`
+
+ [file, filename]
+ end
+
+ def can_generate_thumb?(file)
+ # Some mime type checking
+ end
end
```
-### LocalStorage
+### Versions
-You can use this storage to store files in the local machine running the code. Example:
+Versions is a common and popular feature on other file management libraries, however, they're usually implemented
+in a way that makes the "versioned" attachments behave differently than normal attachments.
+Saviour takes another approach: there's no such concept as a "versioned attachment", there're only attachments.
+The way this works with Saviour is by making one attachment "follow" another one, so that whatever is assigned on
+the main attachment is also assigned automatically to the follower, and when the main attachment is deleted
+also is the follower.
+
+For example:
+
+```ruby
+class Post < ApplicationRecord
+ include Saviour::Model
+
+ attach_file :image do
+ store_dir { "uploads/posts/images/#{model.id}/" }
+ end
+
+ attach_file :image_thumb, follow: :image, dependent: :destroy do
+ store_dir { "uploads/posts/image_thumbs/#{model.id}/" }
+ process_with_file :resize, width: 100, height: 100
+ end
+end
```
-Saviour::Config.storage = Saviour::LocalStorage.new(
- local_prefix: "/var/www/app_name/current/files",
- public_url_prefix: "http://mydomain.com/uploads"
-)
+
+Using the `follow: :image` syntax you declare that the `image_thumb` attachment has to be automatically assigned
+to the same contents as `image` every time `image` is assigned.
+
+The `:dependent` part is mandatory and indicates if the `image_thumb` attachment has to be removed when the
+`image` is removed (with `dependent: :destroy`) or not (with `dependent: :ignore`).
+
+```ruby
+a = Post.create! image: File.open("/path/image.png")
+a.image # => original file assigned
+a.image_thumb # => a thumb over the image assigned
```
-The `local_prefix` option is mandatory, and defines the base prefix under which the storage will store files in the
-machine. You need to configure this accordingly to your use case and deployment strategies, for example, for rails
-and capistrano with default settings you'll need to set it to `Rails.root.join("public/system")`.
+Now, both attachments are independent:
-The `public_url_prefix` is optional and should represent the public endpoint from which you'll serve the assets.
-Same as before, you'll need to configure this accordingly to your deployment specifics.
-You can also assign a Proc instead of a String to dynamically manage this (for multiple asset hosts for example).
+```ruby
+# `image_thumb` can be changed independently
+a.update_attributes! image_thumb: File.open("/path/another_file.png")
-This storage will take care of removing empty folders after removing files.
+# or removed
+a.remove_file_thumb!
+```
+If `dependent: :destroy` has been choosed, then removing `image` will remove `image_thumb` as well:
-### S3Storage
+```ruby
+a.remove_image!
+a.image? # => false
+a.image_thumb? # => false
+````
-An storage implementation using `Fog::AWS` to talk with Amazon S3. Example:
+If the "versioned attachment" is assigned at the same time as the main one, the provided files will be preserved:
+```ruby
+a = Post.create! image: File.open("/path/image.png"), image_thumb: File.open("/path/thumb.jpg")
+a.image # => 'image.png' file
+a.image_thumb # => 'thumb.jpg' file
+
+# The same happens when assignations and db saving are separated:
+
+a = Post.find(42)
+
+# other code ...
+a.image_thumb = File.open("/path/thumb.jpg")
+
+# other code ...
+a.image = File.open("/path/image.png")
+
+# other code ...
+a.save!
+a.image # => 'image.png' file
+a.image_thumb # => 'thumb.jpg' file
```
-Saviour::Config.storage = Saviour::S3Storage.new(
- bucket: "my-bucket-name",
- aws_access_key_id: "stub",
- aws_secret_access_key: "stub"
-)
+
+Finally, even if you selected to use `dependent: :destroy` you may choose to not remove the "versions" when
+removing the main attachment using an extra argument when removing:
+
+```ruby
+a = Post.create! image: File.open("/path/image.png")
+a.remove_image!(dependent: :ignore)
+a.image? # => false
+a.image_thumb? # => true
```
-All passed options except for `bucket` will be directly forwarded to the initialization of `Fog::Storage.new(opts)`,
-so please refer to Fog/AWS [source](https://github.com/fog/fog-aws/blob/master/lib/fog/aws/storage.rb) for extra options.
+The same is true for the opposite, you could use `remove_image!(dependent: :destroy)` if the attachment was
+configured as `dependent: :ignore`.
-The `public_url` method just delegates to the Fog implementation, which will provide the default path to the file,
-for example `https://fake-bucket.s3.amazonaws.com/dest/file.txt`. Custom domains can be configured directly in Fog via
-the `host` option, as well as `region`, etc.
-The `exists?` method uses a head request to verify existence, so it doesn't actually download the file.
+### Transactional behavior
-All files will be created as public by default, but you can set an additional argument when initializing the storage to
-declare options to be used when creating files to S3, and those options will take precedence. Use this for example to
-set an expiration time for the asset. Example:
+When working with attachments inside a database transaction (using Active Record), all the changes made will be
+reverted if the transaction is rolled back.
+On file creation (either creating a new AR model or assigning a file for the first time), the file will be
+available on after save, but will be removed on after rollback.
+
+On file update, changes will be available on after save, but the original file will be restored on after rollback.
+
+On file deletion, the file will be no longer available (via Saviour public api) on after save, but the actual deletion
+will happen on after commit (so in case of rollback the file is never removed).
+
+
+### Concurrency
+
+Saviour will run all processors and storage operations concurrently for all attachments present in a model. For example:
+
+```ruby
+class Product < ApplicationRecord
+ include Saviour::Model
+
+ attach_file :image, SomeUploader
+ attach_file :image_thumb, SomeUploader, follow: :image
+ attach_file :cover, SomeUploader
+end
+
+a = Product.new image: File.open('...'), cover: File.open('...')
+a.save!
```
-Saviour::Config.storage = Saviour::S3Storage.new(
- bucket: "my-bucket-name",
- aws_access_key_id: "stub",
- aws_secret_access_key: "stub",
- create_options: {public: false, 'Cache-Control' => 'max-age=31536000'}
-)
-```
-NOTE: Be aware that S3 has a limit of 1024 bytes for the keys (paths) used. Be sure to truncate to that maximum length
-if you're using an s3 storage, for example with a processor like this:
+At the time that `save!` is executed, 3 threads will be opened. In each one, the processors you defined will be
+executed for that file, and then the result will be written to the storage.
+In case you have so many attachments that processing them concurrently would be undesired you can limit the
+max concurrency with:
+
```ruby
- # http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingMetadata.html
- # Max 1024 bytes for keys in S3
- def truncate_at_max_key_size(contents, filename)
- # Left 20 bytes of margin (up to 1024) to have some room for a name
- if store_dir.bytesize > 1004
- raise "The store_dir used is already bigger than 1004 bytes, must be reduced!"
- end
+Saviour::Config.concurrent_workers = 2
+```
- key = "#{store_dir}#{filename}"
- new_filename = if key > 1024
- # note mb_chars is an active support's method
- filename.mb_chars.limit(1024 - store_dir.bytesize).to_s
- else
- filename
- end
+The default value is 4.
- [contents, new_filename]
- end
-```
+#### stash
-## Source abstraction
+Note that this means **your processor's code must be thread-safe**. Do not issue db queries from processors
+directly, for example. They would be executed in a new connection by AR and you may not be expecting that.
-As mentioned before, you can use `File#assign` with any io like object that responds to `read` and `rewind`. This is already the case for `::File`,
-`Tempfile` or `IO`. Since a file requires also a filename, however, in those cases a random filename will be assigned
-(you can always set the filename using a processor later on).
+Saviour comes with a simple mechanism to gather data from processors so that you can use it later from
+the main thread: `stash`. For example:
-Additionally, if the object responds to `#original_filename` then that will be used as a filename instead of generating
-a random one, or, if the object responds to `#path` then `File.basename(path)` will be used as a name.
-You can create your own classes implementing this API to extend functionality. This library includes two of them: StringSource
-and UrlSource.
+```ruby
+class ImageUploader < Saviour::BaseUploader
+ store_dir { "uploads/thumbs/#{model.id}/#{attached_as}" }
+ process_with_file do |file, filename|
+ width, height = `identify -format "%wx%h" #{Shellwords.escape(file.path)}`.strip.split(/x/).map(&:to_i)
-### StringSource
+ stash(
+ width: width,
+ height: height,
+ size: File.size(file.path)
+ )
-This is just a wrapper class that gives no additional behavior except for implementing the required API. Use it as:
+ [file, filename]
+ end
+ after_upload do |stash|
+ model.update_attributes!(size: stash[:size], width: stash[:width], height: stash[:height])
+ end
+end
```
-foo = Saviour::StringSource.new("my raw contents", "filename.jpg")
-post = Post.find(123)
-post.image = foo
-```
-### UrlSource
+Use `stash(hash)` to push a hash of data from a processor. You can call this multiple times from different processors,
+the hashes you stash will be deep merged. You can then declare an `after_upload` block that will run in the main
+thread once all attachments have been saved to the storage. The block will simply receive the stash hash, and from
+there you can run arbitrary code to persist the info.
-This class implements the source abstraction from a URL. The `read` method will download the given URL and use those
-contents. The filename will be guessed as well from the URL. Redirects will be followed (max 10) and connection retried
-3 times before raising an exception. Example:
+### Dirty tracking
+
+Saviour implements dirty tracking for the attachments. Given the following example:
+
+```ruby
+class User < ApplicationRecord
+ include Saviour::Model
+
+ attach_file(:avatar) do
+ store_dir { "uploads/avatars/#{model.id}/" }
+ end
+end
```
-foo = Saviour::UrlSource.new("http://server.com/path/image.jpg")
-post = Post.find(123)
-post.image = foo
-```
+You can now use:
-## Uploader classes and Processors
+```ruby
+a = User.create! avatar: File.open("avatar.jpg")
-Uploaders are the classes responsible for managing what happens when a file is uploaded into an storage. Use them to define
-the path that will be used to store the file, additional processings that you want to run and versions. See a complete
-example:
+a.avatar = File.open("avatar_2.jpg")
+a.changed? # => true
+
+a.avatar_changed? # => true
+
+a.avatar_was.url # => url pointing to the original avatar.jpg file
+a.avatar_was.read # => previous byte contents
+
+a.changed_attributes # => { avatar: <Saviour::File instance of avatar.jpg>}
+a.avatar_change # => [<Saviour::File instance of avatar.jpg>, <Saviour::File instance of avatar_2.jpg>]
+a.changes # => { avatar: [<Saviour::File instance of avatar.jpg>, <Saviour::File instance of avatar_2.jpg>] }
+
+a.save!
+
+a.avatar_changed? # => false
```
-class ExampleUploader < Saviour::BaseUploader
- store_dir { "/default/path/#{model.id}" }
- process :resize, width: 50, height: 50
- process_with_file do |local_file, filename|
- `mogrify -resize 40x40 #{local_file.path}`
- [local_file, filename]
+
+### AR Validations
+
+You can use `attach_validation` in an Active Record model to declare validations over attachments, for example:
+
+```ruby
+class User < ApplicationRecord
+ include Saviour::Model
+
+ attach_file(:avatar) do
+ store_dir { "uploads/avatars/#{model.id}/" }
end
- process do |contents, filename|
- [contents, "new-#{filename}"]
+ attach_validation :avatar do |contents, filename|
+ errors.add(:avatar, "max 10 Mb") if contents.bytesize > 10.megabytes
+ errors.add(:avatar, "invalid format") unless %w(jpg jpeg).include?(File.extname(filename))
end
+end
+```
- version(:thumb) do
- store_dir { "/default/path/#{model.id}/versions" }
- process :resize, with: 10, height: 10
+Similar as with processors, your block will receive the raw byte contents of the assigned file (or object) and the
+filename. Adding errors is up to the logic you want to have.
+
+Validations can also be expressed as methods in the model:
+
+```ruby
+class User < ApplicationRecord
+ include Saviour::Model
+
+ attach_file(:avatar) do
+ store_dir { "uploads/avatars/#{model.id}/" }
end
- version(:just_a_copy)
+ attach_validation :avatar, :check_format
- def resize(contents, filename, opts)
- # User RMagick to modify contents in memory here
- [contents, filename]
+ def check_format(contents, filename)
+ errors.add(:avatar, "invalid format") unless %w(jpg jpeg).include?(File.extname(filename))
end
end
```
-### store_dir
+In both forms (block or method) an additional 3rd argument will be provided as a hash of `{attached_as: "avatar"}`
+in this example. You can use this to apply different logic per attachment in case of shared validations.
-Use `store_dir` to indicate the default directory under which the file will be stored. You can also use it under a
-`version` to change the default directory for that specific version.
+Those validations will run on before save, so none of the processors you may have defined did run yet. The contents
+and filename provided in the validation are the ones originally assigned to the attachment.
-Note that it's very important that the full path to any attached file to any model is unique. This is typically
-accomplished by using `model.id` and `attached_as` as part of either the `store_dir` or the `filename`, in any
-combination you may want. If this is not satisfied, you may experience unexpected overwrite of files or files
-having unexpected contents, for example if two different models write to the same storage path, and then one
-of them is deleted.
+You can also use the variation `attach_validation_with_file`, which is the same but instead of raw contents you're
+given a `File` object to work with. Use this to preserve memory if that's your use case, same considerations apply
+as in the processor's case.
-### Accessing model and attached_as
+### Introspection
-Both `store_dir` and `process` / `process_with_file` declarations can be expressed passing a block or passing a symbol
-representing a method. In both cases, you can directly access there a method called `model` and a method called
-`attached_as`, representing the original model and the name under which the file is attached to the model.
+Two methods are added to any class including `Saviour::Model` to give you information about what attachments
+have been defined in that class.
-Use this to get info form the model to compose the store_dir, for example, or even to create a processor that
-extracts information from the file and passes this info back to the model to store it in additional db columns.
+`Model.attached_files` will give an array of symbols, representing all the attachments declared in that class.
-### Processors
+`Model.attached_followers_per_leader` will give a hash where the keys are attachments that have versions
+assigned, and the values being an array of symbols, representing the attachments that are following that attachment.
-Processors are the methods (or blocks) that will modify either the file contents or the filename before actually
-upload the file into the storage. You can declare them via the `process` or the `process_with_file` method.
+```ruby
+class Post < ApplicationRecord
+ include Saviour::Model
-They work as a stack, chaining the response from the previous one as input for the next one, and are executed in the
-same order you declare them. Each processor will receive the raw contents and the filename, and must return an array
-with two values, the new contents and the new filename.
+ attach_file :image, SomeUploader
+ attach_file :image_thumb, SomeUploader, follow: :image, dependent: :destroy
+ attach_file :image_thumb_2, SomeUploader, follow: :image, dependent: :destroy
+ attach_file :cover, SomeUploader
+end
-As described in the example before, processors can be declared in two ways:
+Post.attached_files # => [:image, :image_thumb, :image_thumb_2, :cover]
+Post.attached_followers_per_leader # => { image: [:image_thumb, :image_thumb_2] }
+```
-- As a symbol or a string, it will be interpreted as a method that will be called in the current uploader.
- You can optionally set an extra Hash of options that will be forwarded to the method, so it becomes easier to reuse processors.
-- As a Proc, for inline use cases.
+## Extras & Advance usage
-By default processors work with the full raw contents of the file, and that's what you will get and must return when
-using the `process` method. However, since there are use cases for which is more convenient to have a File object
-instead of the raw contents, you can also use the `process_with_file` method, which will give you a Tempfile object,
-and from which you must return a File object as well.
+### Skip processors
-You can combine both and Saviour will take care of synchronization, however take into account that every time you
-switch from one to another there will be a penalty for having to either read or write from/to disk.
-Internally Saviour works with raw contents, so even if you only use `process_with_file`, there will be a penalty at the
-beginning and at the end, for writing and reading to and from a file.
+Saviour has a configuration flag called `processing_enabled` that controls whether or not to execute processors.
+You can set it:
-When using `process_with_file`, the last file instance you return from your last processor defined as
-`process_with_file` will be automatically deleted by Saviour. Be aware of this if you return
-some File instance different than the one you received pointing to a file.
+`Saviour::Config.processing_enabled = false`
-From inside a process you can also access the current store dir with `store_dir`.
+It's thread-safe and can be changed on the fly. Use it if you, for some reason, need to skip processing in a general
+way.
-From inside a process, you can also call `halt_process` to abort the current processing and upload of the file.
-This can be useful for example for an "image_thumb" attachment that can be generic. If you're able to generate a
-thumbnail image for the given file, then it works normally, otherwise halt:
+### Testing
+As file management is an expensive operation if you're working with a remote storage like s3, there
+are some things that you might want to change during test execution.
+
+First of all, you can use a local storage on tests instead of s3, only this will speed up your suite a lot.
+If you have some tests that must run against s3, you can use an s3 spec flag to conditionally
+swap storages on the fly:
+
```ruby
-process_with_file do |file, filename|
- if can_generate_thumbnail?(file) # Only if jpg, png or pdf file
- create_thumbnail(file, filename)
- else
- halt_process
+# config/env/test.rb
+Saviour::Config.storage = ::LocalStorage.new(...)
+
+# spec/support/saviour.rb
+module S3Stub
+ mattr_accessor :storage
+
+ self.storage = Saviour::S3Storage.new(...)
+end
+
+RSpec.configure do |config|
+ config.around(:example, s3_storage: true) do |example|
+ previous_storage = Saviour::Config.storage
+ Saviour::Config.storage = S3Stub.storage
+
+ example.call
+
+ Saviour::Config.storage = previous_storage
end
end
-```
-Finally, processors can be disabled entirely via a configuration parameter. Example:
+it "some regular test" do
+ # local storage here
+end
+it "some test with s3", s3_storage: true do
+ # s3 storage here
+end
```
+
+Finally, you can also choose to disable execution of all processors during tests:
+
+```ruby
+# spec/support/saviour.rb
+
Saviour::Config.processing_enabled = false
-Saviour::Config.processing_enabled = true
```
-You can use this when running tests, for example, or if you want processors to not execute for some reason. The flag can be
-changed in real time and is thread-safe.
+This will skip all processors, so you'll avoid image manipulations, etc. If you have a more complex application
+and you can't disable all processors, but still would want to skip only the ones related to image manipulation,
+I would recommend to delegate image manipulation to a specialized class and then stub all of their methods.
+
+### Sources: url and string
-## Versions
+Saviour comes with two small utility classes to encapsulate values to assign as attachments.
-Versions in Saviour are treated just like an additional attachment. They require you an additional database column to
-persist the file path, and this means you can work with them completely independently of the main file. They can be
-assigned, deleted, etc... independently. You just need to work with the versioned `Saviour::File` instance instead of the main
-one, so for example when assigning a file you'll need to do `object.file(:thumb).assign(my_file)`.
+If you want to provide directly the contents and filename, you can use `Saviour::StringSource`:
-You must create an additional database String column for each version, with the following convention:
+`Post.create! image: Saviour::StringSource.new("hello world", "file.txt")`
-`<attached_as>_<version_name>`
+If you want to assign a file stored in an http endpoint, you can use `Saviour::UrlSource`:
-The only feature versions gives you is following their main file: A version will be assigned automatically if you assign the
-main file, and all versions will be deleted when deleting the main file.
+`Post.create! image: Saviour::UrlSource.new("https://dummyimage.com/600x400/000/fff")`
-In case of conflict, the versioned assignation will be preserved. For example, if you assign both the main file and the version,
-both of them will be respected and the main file will not propagate to the version in this case.
-Defined processors in the Uploader will execute when assigning a version directly. Validations will also execute when assigning
-a version directly (see validation section for details).
+### Custom Storages
-When you open a `version` block within an uploader, you can declare some processors (or change the store dir) only for
-that version. Note that all processors will be executed for every version that exists, plus one time for the base file.
-There are no optimizations done, if your uploader declares one processors first, and from there you open 2 versions,
-the first processors will be executed 3 times.
+An storage is a class that implements the public api expected by Saviour. The abstraction expected
+by Saviour is that, whatever the underlying platform or technology, the storage is able to persist
+the given file using the given path as a unique identifier.
+The complete public api that must be satisfied is:
-## Validations
+- write(raw_contents, path): Given raw byte contents and a full path, the storage is expected to
+persist those contents indexed by the given path, so that later on can be retrieved by the same path.
+The return value is ignored.
-You can declare validations on your model to implement specific checkings over the contents or the filename of an attachment.
+- read(path): Returns the raw contents stored in the given path.
-Take note that validations are executed over the contents given as they are, before any processing. For example you can
-have a validation declaring "max file size is 1Mb", assign a file right below the limit, but then process it in a way that
-increases its size. You'll be left with a file bigger than 1Mb.
+- write_from_file(file, path): Same as write, but providing a file object rather than raw contents. The storage
+has the opportunity to implement this operation in a more performant way, if possible (local storage does here
+a `cp`, for example). The return value is ignored.
-Example of validations:
+- read_to_file(path, file): Same as read, but writing to the given file directly instead of returning raw values.
+The storage has the opportunity to implement this operation in a more performant way, if possible.
+The return value is ignored.
-```
-class Post < ActiveRecord::Base
- include Saviour::Model
- attach_file :image, PostImageUploader
+- delete(path): Removes the file stored at the given path. The return value is ignored.
- attach_validation(:image) do |contents, filename|
- errors.add(:image, "must be smaller than 10Mb") if contents.bytesize >= 10.megabytes
- errors.add(:image, "must be a jpeg file") if File.extname(filename) != ".jpg" # naive, don't copy paste
- end
+- exists?(path): Returns a boolean true/false, depending if the given path is present in the storage or not.
+S3 storage implements this with a HEAD request, for example.
+
+- public_url(path): Returns a string corresponding to an URL under which the file represented by the given
+path is available.
+
+- cp(source_path, destination_path): Copies the file from "source_path" into "destination_path". Overwrites
+"destination_path" if necessary.
+
+- mv(source_path, destination_path): Moves the file from "source_path" into "destination_path". Overwrites
+"destination_path" if necessary, and removes the file at "source_path".
+
+`cp` and `mv` are explicitly created in order to give a chance to the storage to implement the feature in a more
+performant way, for example, s3 implements `cp` as direct copy inside s3 without downloading/uploading the file.
+
+If the given path does not correspond with an existing file, in the case of `read`, `read_to_file`, `delete`, `cp` or
+`mv`, the storage is expected to raise the `Saviour::FileNotPresent` exception.
+
+Any additional information the storage may require can be provided on instance creation (on `initialize`) since
+this is not used by Saviour.
+
+
+### Bypassing Saviour
+
+The only reference to stored files Saviour holds and uses is the path persisted in the database. If you want to,
+you can directly manipulate the storage contents and the database in any custom way and Saviour will just pick
+the changes and work from there.
+
+Since Saviour is by design model-based, there may be use cases when this becomes a performance issue, for example:
+
+##### Bypass example: Nested Cloning
+
+Say that you have a model `Post` that has many `Image`s, and you're working with S3. `Post` has 3 attachments and
+`Image` has 2 attachments. If you want to do a feature to "clone" a post, a simple implementation would be to
+basically `dup` the instances and save them.
+
+However, for a post with many related images, this would represent many api calls and roundtrips to download
+contents and re-upload them. It would be a lot faster to work with s3 directly, issue api calls to copy the
+files inside s3 directly (no download/upload, and even you could issue those api calls concurrently),
+and then assign manually crafted paths directly to the new instances.
+
+
+## FAQ
+
+### how to reuse code in your app, attachment with defaults
+
+If your application manages many file attachments and you want certain things to apply to all of them, you can
+extract common behaviors into a module:
+
+```ruby
+module FileAttachmentHelpers
+ # Shared processors
end
-```
-Validations will always receive the raw contents of the file. If you need to work with a `File` object you'll need to implement
-the necessary conversions.
+module FileAttachment
+ extend ActiveSupport::Concern
-Validations can also be declared passing a method name instead of a block, like this:
+ included do
+ include Saviour::Model
+ end
-```
-class Post < ActiveRecord::Base
- include Saviour::Model
- attach_file :image, PostImageUploader
- attach_validation :image, :check_size
+ class_methods do
+ def attach_file_with_defaults(*args, &block)
+ attached_as = args[0]
- private
+ attach_file(*args) do
+ include FileAttachmentHelpers
- def check_size(contents, filename)
- errors.add(:image, "must be smaller than 10Mb") if contents.bytesize >= 10.megabytes
+ store_dir { "uploads/#{model.class.name.parameterize}/#{model.id}/#{attached_as}" }
+
+ instance_eval(&block) if block
+ process_with_file :sanitize_filename
+ process_with_file :digest_filename
+ process_with_file :truncate_at_max_key_size
+ end
+
+ attach_validation_with_file(attached_as) do |file, _|
+ errors.add(attached_as, 'is an empty file') if ::File.size(file.path).zero?
+ end
+ end
+
+ def validate_extension(*validated_attachments, as:)
+ formats = Array.wrap(as).map(&:to_s)
+
+ validated_attachments.each do |attached_as|
+ attach_validation_with_file(attached_as) do |_, filename|
+ ext = ::File.extname(filename)
+ unless formats.include?(ext.downcase.delete('.'))
+ errors.add(attached_as, "must have any of the following extensions: '#{formats}'")
+ end
+ end
+ end
+ end
end
end
+
+class Post < ApplicationRecord
+ include FileAttachment
+
+ attach_file_with_defaults :cover # Nothing extra needed
+
+ attach_file_with_defaults :image do
+ process_with_file :some_extra_thing
+ end
+end
```
-To improve reusability, validation blocks or methods will also receive a third argument (only if declared in your
-implementation). This third argument is a hash containing `attached_as` and `version` of the validating file.
+In this example we're encapsulating many behaviors that will be given for free to any declared attachments:
+- `store_dir` computed by default into a path that will be different for each class / id / attached_as.
+- 3 generic processors are always run, `sanitize_filename` to ensure we'll have a sane url in the end, `digest_filename` to append a digest and `truncate_at_max_key_size` to ensure we don't reach the 1024 bytes imposed by S3.
+- All attachments will validate that the assigned file must not be empty (0 bytes file).
+- An utility method is added to allow for validations against the filename extension with `validate_extension :image, as: %w[jpg jpeg png]`
-## Active Record Lifecycle integration
-On `after_save` Saviour will upload the changed files attached to the current model, executing the processors as needed.
+### How to manage file removal from forms
-On `after_destroy` Saviour will delete all the attached files and versions.
+This feature can be implemented with a temporal flag in the model, which is exposed in the forms and passed via
+controllers, and a `before_update` to read the value and delete the attachment if present. For example, the
+`FileAttachment` module exposed in the previous point could be extended as such:
-On `validate` Saviour will execute the validations defined.
+```ruby
+module FileAttachment
+ # ...
+ class_methods do
+ def attach_file_with_defaults(*args, &block)
+ attached_as = args[0]
+ # ...
+
+ define_method("remove_#{attached_as}") do
+ instance_variable_get("@remove_#{attached_as}")
+ end
-When validations are defined, the assigned source will be readed only once. On validation time, it will be readed, passed
-to the validation blocks and cached. If the model is valid, the upload will happen from those cached contents. If there
-are no validations, the source will be readed only on upload time, after validating the model.
+ alias_method "remove_#{attached_as}?", "remove_#{attached_as}"
+ define_method("remove_#{attached_as}=") do |value|
+ instance_variable_set "@remove_#{attached_as}", ActiveRecord::Type::Boolean.new.cast(value)
+ end
+
+ before_update do
+ send("remove_#{attached_as}!") if send("remove_#{attached_as}?")
+ end
+ end
+ end
+end
+```
-## FAQ
+Then it can be used as:
-This is a compilation of common questions or features regarding file uploads.
+```ruby
+# This would be a controller code
+a = Post.find(42)
-### Digested filename
+# Params received from a form
+a.update_attributes(remove_image: "t")
+```
-A common use case is to create a processor to include a digest of the file in the filename, in order to automatically
-expire caches. The implementation is left for the user, but a simple example of such processor is this:
-```
- def digest_filename(contents, filename, opts = {})
- separator = opts.fetch(:separator, "-")
+### How to extract metadata from files
- digest = ::Digest::MD5.hexdigest(contents)
- extension = ::File.extname(filename)
+You can use processors to accomplish this. Just be aware that processors run concurrently, so if you want to
+persist you extracted information in the database probably you'll want to use `stash`, see [the section
+about stash feature for examples](#stash).
- new_filename = "#{[::File.basename(filename, ".*"), digest].join(separator)}#{extension}"
- [contents, new_filename]
+### How to process files in background / delayed
+
+As a previous warning note, pushing logic to be run in the background, when they have visible consequences for the application, may
+have undesired side effects and added complexity. For example, as you can't be sure about when the delayed job
+will be completed, your application now needs to handle the uncertainty about the situation: The file processing may
+or may not have run yet.
+
+Implementing a delayed processor means that Saviour is no longer involved in the process. You could add the
+enqueuing of the job when you detect a change in the attachment:
+
+```ruby
+class Post < ApplicationRecord
+ include Saviour::Model
+ attach_file :image
+
+ before_save do
+ if image_changed?
+ # On after commit, enqueue the job
+ end
end
+end
```
+The job then should take the model and the attachment to process and run the processings directly:
+
+```ruby
+a = Post.find(42)
+a.image.with_copy do |f|
+ # manipulate f as desired
+ a.update_attributes! image: f
+end
+```
+
+
### How to recreate versions
-Recreating a version based on the master file can be easily done by just assigning the master file to the version and
-saving the model. You just need a little bit more code in order to preserve the current version filename, for example,
-if that's something you want.
+As "versions" are just regular attachments, you only need to assign to it the contents of the main attachment. You can
+also directly assign attachments between themselves. For example:
-An example service that can do that is the following:
+```ruby
+class Post < ApplicationRecord
+ include Saviour::Model
+ attach_file :image, SomeUploader
+ attach_file :image_thumb, SomeUploader, follow: :image, dependent: :destroy
+end
+
+post = Post.find 42
+post.image_thumb = post.image
+post.save!
```
-class SaviourRecreateVersionsService
- def initialize(model)
- @model = model
- end
- def recreate!(attached_as, *versions)
- base = @model.send(attached_as).read
+### How to digest the filename
- versions.each do |version|
- current_filename = @model.send(attached_as, version).filename
- @model.send(attached_as, version).assign(Saviour::StringSource.new(base, current_filename))
+You can use a processor like this one:
+
+```ruby
+ def digest_filename(file, filename, opts = {})
+ separator = opts.fetch(:separator, '-')
+
+ digest = ::Digest::MD5.file(file.path).hexdigest
+ extension = ::File.extname(filename)
+
+ previous_filename = ::File.basename(filename, '.*')
+
+ if Regexp.new("[0-9a-f]{32}#{Regexp.escape(extension)}$").match(filename)
+ # Remove the previous digest if found
+ previous_filename = previous_filename.split(separator)[0...-1].join(separator)
end
- @model.save!
+ new_filename = "#{previous_filename}#{separator}#{digest}#{extension}"
+
+ [file, new_filename]
end
-end
```
-### Getting metadata from the file
+## License
-### Caching across redisplays in normal forms
-### Introspection (Class.attached_files)
-### Processing in background
+The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
+