Run free audit

robots.txt AI Crawler Checker

Enter a URL and see which AI crawlers your robots.txt allows or blocks. Checks GPTBot (OpenAI training), OAI-SearchBot (ChatGPT Search), ClaudeBot, PerplexityBot, Google-Extended and more. Important: OAI-SearchBot (search/retrieval) and GPTBot (training) are separate - blocking GPTBot does not block ChatGPT Search.

robots.txt is just one of the gates. Want a full AI visibility audit?

Run a free GEO audit
AI Crawler Checklist: Should You Allow or Block GPTBot and ClaudeBot?Sitemaps, lastmod and IndexNow: Getting AI Engines to Index YouIs Your Site Invisible to AI? The 10-Minute Test