# ALL robots will spider the domain
User-agent:*
Disallow:/cgi-bin/
Disallow:/script-folder/
Disallow:/download-folder/
Disallow:/visits.txt/