Crawling and indexing
What is noindex?
noindex is a robots rule, set in a meta tag or an X-Robots-Tag HTTP header, that tells search engines not to show a page in results. The crawler must be able to fetch the page to see the rule, so never block a noindexed page in robots.txt.
How to read it
robots.txt controls fetching; noindex controls showing. A page blocked in robots.txt can still appear in results as a bare URL if other sites link to it, while a fetchable page with noindex will not.
Links on a noindexed page can still be followed, so it can pass crawlers on to pages you do want shown. Check the rule after every template change: a stray noindex on a template removes a whole section from search.
Example from rankecho.io
Open to visitors, out of search
rankecho.io's /benchmark pages carry noindex,follow. Visitors and crawlers can read them and follow their links to the guides, but they do not compete with the guides in Google.
Pages retired for good get a different answer: the old /benchmark/category pages now return 410 Gone, so search engines drop them rather than keep checking a noindex.
Source: rankecho.io's live pages, checked 16 November 2026.
Questions people ask
What is noindex?
noindex is a robots rule, set in a meta tag or an X-Robots-Tag HTTP header, that tells search engines not to show a page in results. The crawler must be able to fetch the page to see the rule, so never block a noindexed page in robots.txt.
Should I block noindexed pages in robots.txt?
No. If the crawler cannot fetch the page, it never sees the noindex, and the URL can stay in results.
How long does noindex take to work?
Until the page is crawled again. URL Inspection can request a recrawl for an urgent page.
Related terms
Read more on rankecho.io
See it on your own site
RankEcho reads your Search Console data and asks AI assistants whether they cite you, then shows each of these numbers for your pages. The audit is free.
Definition and example checked 2026-11-16.
