== CsvPirate Easily create CSVs of any data that can be derived from your models. CsvPirate is the easy way to create a CSV of essentially anything in Rails, in full pirate regalia. It works better if you are wearing a tricorne! My goal is to have it do EVERYTHING it possibly can for me, since almost every project I do needs CSV exports. CsvPirate only works for commissions of swag OR grub! Initialize method (a.k.a new()) takes a hash of params: # CsvPirate only works for commissions of swag OR grub! # :swag the ARrr collection of swag to work on (optional) # :grub the ARrr class that the spyglasses will be used on (optional) # :spyglasses named scopes in your model that will refine the rows in the CSV according to conditions of the spyglasses, # and order them according to the order of the spyglasses (optional) # :booty booty (columns/methods) on your model that you want printed in the CSV, also used to create the figurehead (CSV header) # :chart array of directory names (relative to rails root if using rails) which will be the filepath where you want to hide your loot # :wagonner name of document where you will give detailed descriptions of the loot # :aft filename extention ('.csv') # :shrouds CSV column separator, default is ','. For tsv, tab-delimited, "\t" # :chronometer keeps track of when you hunt for treasure # :gibbet filename spacer after the date, and before the iterative counter/timestamp. MuST contain a '.' # :swab can be :counter, :timestamp, or :none # :counter - default, each successive run will create a new file using a counter # :timestamp - each successive run will create a new file using a HHMMSS time stamp # :none - no iterative file naming convention, just use waggoner and aft # :mop can be :clean or :dirty (:overwrite or :append) (only has an effect if :swab is :none) since overwriting is irrelevant for a new file # :clean - do not use :counter or :timestamp, and instead overwrite the file # :dirty - do not use :counter, or :timestamp, or :overwrite. Just keep adding on. # :bury_treasure should we store the csv data as it is collected in an array in Ruby form for later use (true), or just write the CSV (false)? # check the source to see if there anything else hiding in there! The create method has the same parameters, and actually creates the CSV. Avast! Here be pirates! To brush up on pirate coding naming conventions: http://www.privateerdragons.com/pirate_dictionary.html Perhaps the next version will have CSV importing abilities... == On The Web Source: http://github.com/pboling/csv_pirate Release Announcement: http://galtzo.blogspot.com/2009/03/csv-pirate.html == Installation Gem from Gemcutter (after setting up your machine to use the gemcutter repo, see http://gemcutter.org): sudo gem install csv_pirate Gem Using Git building from source: mkdir -p ~/src cd ~/src git clone git://github.com/pboling/csv_pirate.git cd csv_pirate gem build csv_pirate.gemspec sudo gem install csv_pirate-3.3.0.gem # (Or whatever version gets built) Then cd to your rails app to optionally freeze the gem into your app: rake gems:freeze GEM=csv_pirate Installing Gem from Github's Gem Server: sudo gem install pboling-csv_pirate -s http://gems.github.com Plugin using Git: ruby script/plugin install git://github.com/pboling/csv_pirate.git If installed as a gem load it with config.gem 'csv_pirate' # or 'pboling-csv_pirate' if from github == Upgrading In older versions :chart was a string which indicated where you wanted to hide the loot (write the csv file) Now it must be an array of directory names. So if you want your loot in "log/csv/pirates/model_name", then chart is: ['log','csv','pirates','model_name'] CsvPirate ensures that whatever you choose as your chart exists in the filesystem, and creates the directories if need be. == Usage with ActiveRecord What's the simplest thing that'll work? class MyClass < ActiveRecord::Base has_csv_pirate_ship # defaults to csv of all columns of all records end MyClass.blindfold # creates the csv, and returns the CsvPirate instance MyClass.walk_the_plank # creates the csv, and returns contents of the exported data (that was written into the csv) (as a string) MyClass.land_ho # Does Not create the csv, sets up the CsvPirate instance. You can manipulate it and then call .hoist_mainstay on it to create the csv == Usage without ActiveRecord Since the defaults assume an active record class you need to override some of them: class Star extend NinthBit::PirateShip::ActMethods has_csv_pirate_ship :booty => [:name, :distance, :spectral_type, {:name => :hash}, {:name => :next}, {:name => :upcase}, :star_vowels], :spyglasses => [:get_stars] attr_accessor :name, :distance, :spectral_type def initialize(*args) @name = args.first[:name] @distance = args.first[:distance] @spectral_type = args.first[:spectral_type] end def star_vowels self.name.tr('aeiou', '*') end def self.get_stars [ Star.new(:name => "Proxima Centauri", :distance => "4.2 LY", :spectral_type => "M5.5Vc"), Star.new(:name => "Rigil Kentaurus", :distance => "4.3 LY", :spectral_type => "G2V"), Star.new(:name => "Barnard's Star", :distance => "5.9 LY", :spectral_type => "M3.8V"), Star.new(:name => "Wolf 359", :distance => "7.7 LY", :spectral_type => "M5.8Vc"), Star.new(:name => "Lalande 21185", :distance => "8.26 LY", :spectral_type => "M2V"), Star.new(:name => "Luyten 726-8A and B", :distance => "8.73 LY", :spectral_type => "M5.5 de & M6 Ve"), Star.new(:name => "Sirius A and B", :distance => "8.6 LY", :spectral_type => "A1Vm"), Star.new(:name => "Ross 154", :distance => "9.693 LY", :spectral_type => "M3.5"), Star.new(:name => "Ross 248", :distance => "10.32 LY", :spectral_type => "M5.5V"), Star.new(:name => "Epsilon Eridani", :distance => "10.5 LY", :spectral_type => "K2V") ] end end rails development >a = Star.blindfold => #:hash}, {:name=>:next}, {:name=>:upcase}, :star_vowels], @bury_treasure=false, @swab=:counter, @chronometer=Sun, 04 Oct 2009, @maroon="name,distance,spectral_type,namehash,namenext,nameupcase,star_vowels\nProxima Centauri,4.2 LY,M5.5Vc,971295636,Proxima Centaurj,PROXIMA CENTAURI,Pr*x*m* C*nt**r*\nRigil Kentaurus,4.3 LY,G2V,-231389024,Rigil Kentaurut,RIGIL KENTAURUS,R*g*l K*nt**r*s\nBarnard's Star,5.9 LY,M3.8V,1003964756,Barnard's Stas,BARNARD'S STAR,B*rn*rd's St*r\nWolf 359,7.7 LY,M5.8Vc,429493790,Wolf 360,WOLF 359,W*lf 359\nLalande 21185,8.26 LY,M2V,466625069,Lalande 21186,LALANDE 21185,L*l*nd* 21185\nLuyten 726-8A and B,8.73 LY,M5.5 de & M6 Ve,-886693495,Luyten 726-8A and C,LUYTEN 726-8A AND B,L*yt*n 726-8A *nd B\nSirius A and B,8.6 LY,A1Vm,-969980943,Sirius A and C,SIRIUS A AND B,S*r**s A *nd B\nRoss 154,9.693 LY,M3.5,-26506942,Ross 155,ROSS 154,R*ss 154\nRoss 248,10.32 LY,M5.5V,-18054910,Ross 249,ROSS 248,R*ss 248\nEpsilon Eridani,10.5 LY,K2V,931307088,Epsilon Eridanj,EPSILON ERIDANI,Eps*l*n Er*d*n*\n", @waggoner="Star", @astrolabe=false, @grub=Star, @rhumb_lines=#, @nocturnal="Star.20091004.export.3.csv", @aft=".csv", @gibbet=".export", @shrouds=",", @swag=[#, #, #, #, #, #, #, #, #, #], @chart=["log", "csv"]> rails development >a.weigh_anchor name,distance,spectral_type,namehash,namenext,nameupcase,star_vowels Proxima Centauri,4.2 LY,M5.5Vc,971295636,Proxima Centaurj,PROXIMA CENTAURI,Pr*x*m* C*nt**r* Rigil Kentaurus,4.3 LY,G2V,-231389024,Rigil Kentaurut,RIGIL KENTAURUS,R*g*l K*nt**r*s Barnard's Star,5.9 LY,M3.8V,1003964756,Barnard's Stas,BARNARD'S STAR,B*rn*rd's St*r Wolf 359,7.7 LY,M5.8Vc,429493790,Wolf 360,WOLF 359,W*lf 359 Lalande 21185,8.26 LY,M2V,466625069,Lalande 21186,LALANDE 21185,L*l*nd* 21185 Luyten 726-8A and B,8.73 LY,M5.5 de & M6 Ve,-886693495,Luyten 726-8A and C,LUYTEN 726-8A AND B,L*yt*n 726-8A *nd B Sirius A and B,8.6 LY,A1Vm,-969980943,Sirius A and C,SIRIUS A AND B,S*r**s A *nd B Ross 154,9.693 LY,M3.5,-26506942,Ross 155,ROSS 154,R*ss 154 Ross 248,10.32 LY,M5.5V,-18054910,Ross 249,ROSS 248,R*ss 248 Epsilon Eridani,10.5 LY,K2V,931307088,Epsilon Eridanj,EPSILON ERIDANI,Eps*l*n Er*d*n* => # == Advanced Usage & Examples Assuming a Make (as in manufacturers of automobiles) model like this: # == Schema Information # # Table name: makes # # id :integer(4) not null, primary key # name :string(255) # factory :string(255) # sales :integer(4) # class Make < ActiveRecord::Base has_many :vehicle_models named_scope :factory_in_germany, :conditions => ["factory = ?", "Germany"] # Showing all available options with their default values has_csv_pirate_ship :chart => ['log','csv'] # Array of Strings: directory where csv will be created (Yes, it creates all of them if they do not already exist) :aft => '.csv' # String: filename extension, usually this would be '.csv', but can be whatever you want. :gibbet => '.export' # String: Middle part of the filename {the '.' is required for iterative filenames, set :swab => :none to turn off iterative filenames} # Comes after waggoner and chronometer, before swabbie and aft :waggoner => "#{Make}" # String: First part of filename # Must provide :swag or :grub (not both) :swag => nil # Array of objects: to use to create the CSV (i.e. you've already done the query and have the results you want a CSV of) :grub => Make # Class: on which to call the method chain in :spyglasses that will return the array of objects to be placed in :swag by CsvPirate (See description of swag above). :spyglasses => [:all] # Array of symbols/strings: Methods that will be chained together and called on :grub in order to get the :swag records which will become the rows of the CSV. :booty => Make.column_names # Array of symbols/strings or nested hashes of symbols/strings: Methods to call on each object in :swag. These become the columns of the CSV. The method names become the CSV column headings. Methods can be chained to dig deep (e.g. traverse several ActiveRecord associations) to get at a value for the CSV. :swab => :counter # Symbol: What kind of file counter to use to avoid overwtiting the CSV file, :counter is Integer, :timestamp is HHMMSS, :none is no file counter, increasing the liklihood of duplicate filenames on successive csv exports. :mop => :clean # Symbol: If we DO end up writing to a preexisting file (by design or accident) should we overwrite (:clean) or append (:dirty)? :shrouds => ',' # String: Delimiter for CSV. '\t' will create a tab delimited file (tsv), '|' will create a pipe delimited file. :bury_treasure => true # Boolean: Should the array of objects in :swag be stored in the CsvPirate object for later inspection? # A customized version to create a tab delimited file for this class might look like this: # has_csv_pirate_ship :spyglasses => [:factory_in_germany], # :booty => [:id, :name], # :shrouds => '\t' # # kepping the rest of the options at the default values, so they don't need to be defined. end To create a csv of the names and ids of makes with factories in germany and return the text of the export: Make.walk_the_plank # Get it? HA! If you can't believe I wrote this whole thing JUST to be able to make jokes like that... check ma sources :) The name of the csv that comes out will be (by defualt located in log directory): Make.20090930.export.13.csv Where Make is the class, 20090930 is today's date, .export is the gibbet, and 13 is the iterative file counter, meaning I've run this export 13 times today. All of those filename parts are customizable to a degree. For example if you want to have the date NOT be today's date you can supply your own date: Make.walk_the_plank({:chronometer => Date.parse("December 21, 2012") }) # Filename would be: Make.20121221.export.13.csv Make.walk_the_plank({:chronometer => false }) # Filename would be: Make.export.13.csv What if you want the file name to be always the same and to always append to the end of it? #Example: I want the file to be named "data", with no extension, both of the following accomplish that: Make.walk_the_plank({:chronometer => false, :gibbet => "", :aft => "", :swab => :none, :waggoner => 'data'}) Make.blindfold(:chronometer => false, :gibbet => "", :aft => "", :swab => :none, :waggoner => 'data') All of the options to has_csv_pirate_ship are available to walk_the_plank, land_ho, and blindfold, as well as to the raw class methods CsvPirate.new and CsvPirate.create, but not neccessarily the other way around. You can also customize the CSV, for example if you want to customize which columns are in the csv: Make.walk_the_plank({:booty => [:id, :name, :sales]}) You want a timestamp file counter instead of the integer default: Make.walk_the_plank({:booty => [:id, :name, :sales], :swab => :timestamp}) If you want to append each export to the end of the same file (on a per day basis): Make.walk_the_plank({:booty => [:id, :name, :sales], :spyglasses => [:all], :swab => :none, :mop => :dirty}) If you want to restrict the csv to a particular set of named scopes: Make.walk_the_plank({:booty => [:id, :name, :sales], :spyglasses => [:with_abs, :with_esc, :with_heated_seats]}) If you want to create a tsv (tab-delimited) or psv (pipe-delimited) instead of a csv: Make.walk_the_plank({:booty => [:id, :name, :sales], :shrouds => '\t'}) Make.walk_the_plank({:booty => [:id, :name, :sales], :shrouds => '|'}) If you have a method in the Make class like this: def to_slug "#{self.name}_#{self.id}" end getting it in the CSV is easy peasy: Make.walk_the_plank({:booty => [:id, :name, :to_slug]}) If you want to traverse Active Record Associations, or call a method on the return value of another method (unlimited nesting): Make.walk_the_plank({:booty => [:id, :name, :to_slug, {:to_slug => :hash}]}) #will call .hash on the result of make.to_slug Make.walk_the_plank({:booty => [:id, :name, :to_slug, {:to_slug => {:hash => :abs}}]}) #returns make.to_slug.hash.abs Add whatever methods you want to the :booty array. Write new methods, and add them! Make lots of glorious CSVs full of data to impress the pointy ones in the office. You can also use the raw CsvPirate class itself directly wherever you want. The following two sets of code are identical: csv_pirate = CsvPirate.new({ :grub => User, :spyglasses => [:active,:logged_in], :waggoner => 'active_users_logged_in', :booty => ["id","number","login","created_at"], :chart => ['log','csv'] }) csv_pirate.hoist_mainstay() # creates CSV file and writes out the rows CsvPirate.create({ :grub => User, :spyglasses => [:active,:logged_in], :waggoner => 'active_users_logged_in', :booty => ["id","number","login","created_at"], :chart => ['log','csv'] })# creates CSV file and writes out the rows Another example using swag instead of grub: users = User.logged_out.inactive csv_pirate = CsvPirate.new({ :swag => users, :waggoner => 'inactive_users_not_logged_in', :booty => ["id","number","login","created_at"], :chart => ['log','csv'] }) csv_pirate.hoist_mainstay() Then if you want to get your hands on the loot immediately: csv_pirate.weigh_anchor() For those who can't help but copy/paste into console and then edit: csv_pirate = CsvPirate.new({:grub => User,:spyglasses => [:active,:logged_in],:waggoner => 'active_users_logged_in',:booty => ["id","number","login","created_at"],:chart => ['log','csv']}) OR csv_pirate = CsvPirate.new({:swag => users,:waggoner => 'inactive_users_not_logged_in',:booty => ["id","number","login","created_at"],:chart => ['log','csv']}) == Downloading the CSV You have the same Make class as above, and you have a MakeController: class MakeController < ApplicationController def download_csv csv_pirate = Make.blindfold send_data csv_pirate.maroon, :type => 'text/csv; charset=iso-8859-1; header=present', :disposition => "attachment; filename=#{csv_pirate.nocturnal}" end end == Advanced Example with Nested Methods You have a VehicleModel class and the same Make class as up above: # == Schema Information # # Table name: vehicle_models # # id :integer(4) not null, primary key # name :string(255) # year :integer(4) # horsepower :integer(4) # price :integer(4) # electric :boolean(1) # make_id :integer(4) # class VehicleModel < ActiveRecord::Base belongs_to :make has_csv_pirate_ship :booty => [:id, :name, :year, {:make => :name}, {:tires => {:size => {:width => :inches}}}] def tires; TireSize.new; end end class TireSize # To call an instance method you need to return an instance def size; TireWidth.new; end end class TireWidth # To call a class method you need to return the class object def width; Measurement; end end class Measurement def self.inches; 13; end end Then to create the CSV: a = VehicleModel.blindfold Then check the output from the console: a.weigh_anchor id,name,year,makename,tiressizewidthinches 1,Cavalier,1999,Chevrolet,13 2,Trailblazer,2006,Chevrolet,13 3,Corvette,2010,Chevrolet,13 4,Mustang,1976,Ford,13 5,Lebaron,1987,Chrysler,13 6,Avalon,1996,Toyota,13 => # Joy to recursive code everywhere! If you wanted to create the CsvPirate object and then modify it before creating the csv you can do that too. Does not actually create the csv, so you need to do this in your code: csv_pirate = VehicleModel.land_ho({:booty => [:id, :name, :year, :horsepower, :price]}) This allows you to modify the csv_pirate object before creating the csv like this: csv_pirate.booty -= [:id, :name] csv_pirate.hoist_mainstay() == Thanks Thanks go to 9thBit LLC (http://www.9thBit.net) Sagebit LLC (http://www.sagebit.com) - Sagebit has been very supportive of my open source efforts, thanks Ben! TimePerks LLC (http://www.timeperks.com) - Many useful enhancements were requested and paid for by TimePerks ---------------------------------------------------------------------------------- Author: Peter Boling, peter.boling at gmail dot com Copyright (c) 2009 Peter H. Boling of 9thBit LLC, released under the MIT license. See LICENSE for details.