A Simple Verified Robots.txt
Use this to block AI crawlers. Google Search stays on.
Disclaimer: I am aware that I’m using the robots.txt generator at my own risk. No liability shall be accepted by Sphere for any errors or non-indexing of a website.
How to install:
- Download the robots.txt file
- Upload to your website root (where your homepage is)
- Verify its correct with the tool below
Audit your current robots.txt?
What this does
Blocks 100+ AI crawlers (GPTBot, ClaudeBot, Google-Extended, etc.) while explicitly allowing search engines like Google, Bing, and DuckDuckGo.
Is it safe?
Yes. This is the standard method used by major publishers. Your SEO and Google rankings won't be affected and it's completely free
Will they respect it?
Almost all AI companies follow robots.txt directives (OpenAI, Anthropic, Google, etc.). See which bots respect robots.txt .
Bot list maintained at github.com/ai-robots-txt