# /robots.txt file User-agent: * Disallow: /includes Disallow: /cgi-bin Disallow: /xml Disallow: /send_commentary Disallow: /cms Disallow: /flash Disallow: /partner_logo Disallow: /robo_test Disallow: /test Disallow: /column_images Crawl-delay: 10 User-agent: Slurp Crawl-delay: 10 #Sitemap: http://www.project-syndicate.org/sitemap.xml.gz