Free SEO tool · SEOByIndia
Free Robots.txt and XML Sitemap Checker
Check the crawler document the server actually returns. Use the full URL for a public robots.txt file or XML sitemap. The checker shows the HTTP response, an excerpt and any sitemap declarations it can read, so you can separate a missing document from a rule that needs review.
YOUR WORKSPACE
Robots & sitemap check
How to use it
- Use the correct protocol and host. For robots.txt, check the path at the root of the relevant host.
- Read the response and identify whether it contains the document you expected. If the robots file declares a sitemap, open that exact URL next.
- For a sitemap index, select a child file and sample a few listed URLs. Confirm those addresses are the intended public, canonical destinations.
Worked example
A launch team sees a sitemap link in robots.txt that still points to a retired hostname. The checker helps identify the published declaration. Update the sitemap reference to the maintained file, then verify both the robots response and the new sitemap URL.
How to read the result
This is a live document fetch. Rule matching for a specific crawler and path requires checking the applicable user-agent group, and indexing depends on further signals. Use Search Console to review Google’s sitemap processing for a property you can access.
