Contents

# advertising-related bots:
User-agent: Mediapartners-Google*
Disallow: /

# Wikipedia work bots:
User-agent: IsraBot
Disallow:

Disallow: /admin
Allow: /public/home

Version data entries

17 entries across 17 versions & 1 rubygems

Version	Path
cms_scanner-0.0.18	spec/fixtures/finders/interesting_files/robots_txt/robots.txt
cms_scanner-0.0.17	spec/fixtures/finders/interesting_files/robots_txt/robots.txt
cms_scanner-0.0.16	spec/fixtures/interesting_files/robots_txt/robots.txt
cms_scanner-0.0.15	spec/fixtures/interesting_files/robots_txt/robots.txt
cms_scanner-0.0.14	spec/fixtures/interesting_files/robots_txt/robots.txt
cms_scanner-0.0.13	spec/fixtures/interesting_files/robots_txt/robots.txt
cms_scanner-0.0.12	spec/fixtures/interesting_files/robots_txt/robots.txt
cms_scanner-0.0.11	spec/fixtures/interesting_files/robots_txt/robots.txt
cms_scanner-0.0.10	spec/fixtures/interesting_files/robots_txt/robots.txt
cms_scanner-0.0.9	spec/fixtures/interesting_files/robots_txt/robots.txt
cms_scanner-0.0.8	spec/fixtures/interesting_files/robots_txt/robots.txt
cms_scanner-0.0.7	spec/fixtures/interesting_files/robots_txt/robots.txt
cms_scanner-0.0.6	spec/fixtures/interesting_files/robots_txt/robots.txt
cms_scanner-0.0.5	spec/fixtures/interesting_files/robots_txt/robots.txt
cms_scanner-0.0.4	spec/fixtures/interesting_files/robots_txt/robots.txt
cms_scanner-0.0.3	spec/fixtures/interesting_files/robots_txt/robots.txt
cms_scanner-0.0.2	spec/fixtures/interesting_files/robots_txt/robots.txt