# Wiki-spammers use search engines to find wiki sandboxes to crap in # also web stats generators to do referrer spamming. The blocking of # phpBB is because search engines hit on the sid= attrib which # causes a LOT of excess requests. # Also blocking the image gallery, because there's a lot of them and there # is no need for it to be indexed User-agent: * Disallow: /Main/WikiSandbox Disallow: /webalizer Disallow: /VgaPlanets Disallow: /pbem Disallow: /oursforum Disallow: /albums Disallow: /websvn Disallow: /ourslib # I don't really like the idea of NPBot User-agent: NPBot Disallow: / # MJ12Bot behaves badly with gallery SIDs User-agent: MJ12bot Disallow: /gallery