Free tool · no signup

robots.txt tester.
Including the AI crawlers.

Paste your robots.txt, pick a crawler, see which URLs are allowed or blocked — with the exact rule that decided. Covers GPTBot, ClaudeBot, Perplexity, Google-Extended, and 8 more, not just the legacy search bots.

4 group(s) · 7 rule(s) · 1 sitemap(s)
Verdict
Allowed

Googlebot requesting /admin has no matching rule in the "googlebot" group, so the default (allow) applies.

All known crawlers vs this URL
  • Googlebotsearchallow
  • Bingbotsearchdisallow · /admin
  • DuckDuckBotsearchdisallow · /admin
  • Yahoo Slurpsearchdisallow · /admin
  • YandexBotsearchdisallow · /admin
  • Baiduspidersearchdisallow · /admin
  • GPTBotAIdisallow · /
  • ClaudeBotAIdisallow · /
  • PerplexityBotAIdisallow · /admin
  • Google-ExtendedAIdisallow · /admin
  • CCBot (Common Crawl)AIdisallow · /admin
  • Facebook crawlersocialdisallow · /admin
Sitemaps declared
  • https://vyrable.ai/sitemap.xml

Worried the AI crawlers are reading something they shouldn't? Vyrable's AI Visibility tracker shows you what models actually say about your brand — whether or not you want them to.