robots.txt Syntax Cheat Sheet — All Directives Explained
Complete robots.txt syntax reference: User-agent, Allow, Disallow, Crawl-delay, Sitemap wildcards (* and $), and common patterns. Copy-paste ready.
Validate your robots.txt live — detect all issues and get the corrected file.
Open robots.txt Validator →