Sha256: d18a02dd417ee17b8356924d977d24d8bf1b834085ecf37e1a15c09976a01846
Contents?: true
Size: 572 Bytes
Versions: 8
Compression:
Stored size: 572 Bytes
Contents
= Anemone == DESCRIPTION Anemone is a web spider framework that can spider a domain and collect useful information about the pages it visits. It is versatile, allowing you to write your own specialized spider tasks quickly and easily. == FEATURES * Multi-threaded design for high performance * Tracks 301 HTTP redirects to understand a page's aliases * Built-in BFS algorithm for determining page depth * Allows exclusion of URLs based on regular expressions == REQUIREMENTS * nokogiri == EXAMPLES See the +bin+ directory for several examples of useful Anemone tasks.
Version data entries
8 entries across 8 versions & 3 rubygems