Live audit
Robots and Meta Tester
Test crawl access and index directives together by checking robots.txt, meta robots, and X-Robots-Tag settings.
Step 1: Fetch URL Data
Step 2: Review and Test
Crawl Access
-
Matched Group
-
Meta Index
-
Meta Follow
-
Fetch a URL to begin
Enter a page URL and click fetch to pull robots.txt, meta robots, and X-Robots-Tag values.