test/test_app/public/robots.txt in pauldowman-ec2onrails-0.9.10 vs test/test_app/public/robots.txt in pauldowman-ec2onrails-0.9.10.0
- old
+ new
@@ -1,4 +1,8 @@
-# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
\ No newline at end of file
+# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
+#
+# To ban all spiders from the entire site uncomment the next two lines:
+# User-Agent: *
+# Disallow: /