User-agent: www.competitor-analysis.com Disallow: / User-agent: * Disallow: # /robots.txt file for http://www.zrd.com/ # mail webmaster for constructive feedback # The first paragraph has all relative URLs starting with '/' disallowed to competitor-analysis # Because all relative URL's on a server start with '/', this means the entire site is closed off to them # The second paragraph indicates that all other robots are allowed to visit