An XML sitemap is a structured file (usually at https://example.com/sitemap.xml) that tells search engines which pages exist on a site, when they were last updated, and how often they change. It is one of the most reliable ways to ensure crawlers discover all important pages, particularly on large sites or sites with thin internal linking.
What to include: All canonical, indexable URLs — pages that are not noindexed, not blocked by robots.txt, and not duplicates.
What NOT to include:
- Noindexed pages — a conflict between noindex and sitemap listing confuses crawlers
- Redirecting URLs — list only the final destination
- Canonicalised duplicate URLs — only the canonical should appear
- Admin, login, or private pages
Sitemap index: Large sites use a sitemap index file pointing to multiple individual sitemap files, each with a 50,000 URL limit.
What SearchVitals checks: The Technical SEO audit fetches and validates the XML sitemap, checking for broken links, noindexed pages listed within it, non-200 status codes, and whether the sitemap is referenced in robots.txt.