A noindex directive tells Google and other search engines: "crawl this page if you want, but do not add it to your search index and do not show it in search results." It is the standard way to hide pages from search results without blocking crawl access entirely.
Two implementation methods:
- HTML meta tag (in the
<head>):<meta name="robots" content="noindex"> - HTTP response header:
X-Robots-Tag: noindex
When to use noindex: Thank-you pages, internal search results, staging pages accidentally reachable publicly, and pages with thin or duplicate content that should not compete in search.
Common mistake: Applying noindex too broadly — for example, through a CMS setting that adds noindex to an entire category — accidentally removes important pages from the index.
What SearchVitals checks: The Crawl & Index audit scans all crawled pages for noindex directives and cross-references them against the XML sitemap. Pages that are noindexed but listed in the sitemap are flagged as a conflict requiring resolution.