AI Crawler Access Audit
See which AI and search crawlers (GPTBot, ClaudeBot, Google-Extended, CCBot and 150+ more) each site allows or blocks in robots.txt, and whether llms.txt is published. One row per URL, at most two small requests per site, no page content.
25%
users