Definition
ClaudeBot is Anthropic's web crawler, identified by the ClaudeBot user-agent string. It collects publicly available web content that may be used to improve Anthropic's Claude models. Unlike OpenAI, which splits training (GPTBot) and search (OAI-SearchBot) into separate crawlers, Anthropic currently operates ClaudeBot as a single crawler. Blocking ClaudeBot via robots.txt prevents Anthropic from indexing your content for training purposes. As of mid-2026, Claude's web access features (search and citations) rely on third-party search APIs rather than ClaudeBot's own index, so blocking ClaudeBot does not immediately remove you from Claude's search answers — but this architecture could change. For GEO decision-making, the practical stance for most brands is to allow ClaudeBot: the training-data inclusion increases the chance Claude mentions your brand in answers from its parametric knowledge, and the privacy risk is limited to content that is already publicly accessible.