# Robots.txt - All robots will spider the domain User-agent: * Disallow: /Ajax/ Disallow: /dns_admin/ Disallow: /domreg/ Disallow: /Helpdesk/ Disallow: /includes/ Disallow: /lang/ Disallow: /Library/ Disallow: /mail_admin/ Disallow: /Private/ Disallow: /scripts/