# $Id$ # robots.txt RFC: http://www.robotstxt.org/wc/norobots-rfc.html # Validated by http://www.searchengineworld.com/cgi-bin/robotcheck.cgi User-agent: * # Crawl-delay: 1 Disallow: /cgi-bin/ Disallow: /backup/ Disallow: /twiki/ Disallow: /test/