Free Tool

Robots.txt Builder and Tester

Build a robots.txt file that lets AI crawlers like GPTBot, ClaudeBot, and PerplexityBot discover your content — while keeping private pages out of reach.

Start from a template

Pick a starting point that matches your site type. You can customize everything after.

or

Paste your existing robots.txt

Import your current file to edit it visually and test rules against specific crawlers.

Want to see if AI crawlers are actually reaching your site?

ChatRank tracks your brand's visibility across ChatGPT, Perplexity, and Google AI — showing you which pages get cited and which are invisible.