User-agent: * Disallow: /cgi-bin/ Disallow: /icons/ Disallow: /badbot_catcher/ Disallow: /systemstatus/ Disallow: /mailman/ Disallow: /skydiving/ Disallow: /cache/ Disallow: /inc/ Disallow: /img/ Disallow: /css/ Disallow: /media/ # Baidu is a bad ... du User-agent: Baiduspider Disallow: / # Rover is a bad dog User-agent: Roverbot Disallow: / # EmailSiphon is a hunter/gatherer which extracts email addresses for # spam-mailers to use User-agent: EmailSiphon Disallow: / User-agent: NPBot Disallow: / User-agent: psbot Disallow: / # just kinda bugs me User-agent: ia_archiver Disallow: / User-agent: LinkWalker Disallow: / User-agent: CyberPatrol Disallow: /