Glossary
What is noindex?
Noindex is an instruction, in a robots meta tag or an X-Robots-Tag HTTP header, that tells search engines not to include a page in their index. The page must stay crawlable so the engine can see the instruction. It's the right tool for keeping pages like logins, thank-you pages or thin pages out of results.
Two ways to set it
- Meta tag:
<meta name="robots" content="noindex">in the page head. - HTTP header:
X-Robots-Tag: noindex, useful for PDFs and non-HTML files, or whole sections set at the server or CDN.
When to use it
- Login, account and checkout pages.
- Internal search results and filtered pages.
- Thank-you and confirmation pages.
- Thin or duplicate pages you need for users but not for search.
Noindex vs robots.txt
| noindex | robots.txt Disallow | |
|---|---|---|
| Stops crawling | No | Yes |
| Keeps out of results | Yes | Not reliably |
| Must be crawlable | Yes | N/A |
Combining them is a classic mistake: if robots.txt blocks the page, the crawler never sees the noindex.
Example
We serve the app's private routes with an X-Robots-Tag: noindex header at the edge, while public pages like this glossary are indexable and listed in sitemaps.
FAQ
How long until a noindexed page drops out?
After the next crawl; request recrawl in Search Console to speed it up.
Does noindex pass link equity?
Links on a noindexed page may be followed for a while; over time Google treats long-term noindex pages as nofollow too.
Should I noindex thin programmatic pages?
Improve them if they're useful; noindex the ones that aren't.
Sources
- Google Search Central: Block indexing with noindex
- Google Search Central: Robots meta tag specifications