Sha256: 28bdb4792e0705564b4776c81f4efa05f0aba6896fa5ea821ddf36d5e5d51926
Contents?: true
Size: 851 Bytes
Versions: 1
Compression:
Stored size: 851 Bytes
Contents
require 'ostruct' require 'anemone/core' module Anemone # Version number VERSION = '0.0.2' # User-Agent string used for HTTP requests USER_AGENT = "Anemone/#{self::VERSION}" #module-wide options def Anemone.options=(options) @options = options end def Anemone.options @options end # # Convenience method to start a crawl using Core # def Anemone.crawl(url, options = {}, &block) Anemone.options = OpenStruct.new(options) #by default, run 4 Tentacle threads to fetch pages Anemone.options.threads ||= 4 #disable verbose output by default Anemone.options.verbose ||= false #by default, throw away the page response body after scanning it for links, to save memory Anemone.options.discard_page_bodies ||= true Core.crawl(url, &block) end end
Version data entries
1 entries across 1 versions & 1 rubygems
Version | Path |
---|---|
anemone-0.0.2 | lib/anemone/anemone.rb |