noindex

Definition

noindex is an instruction to search engines that a URL should not appear in search results. Google describes it as a rule set either with a <meta> tag or an HTTP response header. When Googlebot crawls the page and extracts the rule, Google drops the page from Search entirely, regardless of whether other sites link to it.

The two forms have the same effect:

<meta name="robots" content="noindex">

X-Robots-Tag: noindex

Using name="googlebot" instead of name="robots" limits the rule to Google's crawlers. The header form, covered under X-Robots-Tag, also works for PDFs, images and other non-HTML files.

In practice

The most common surprise is the interaction with robots.txt. A noindex rule only works if Googlebot can fetch the page. If robots.txt disallows the URL, the rule is never read, and the bare URL can still be indexed from external links. Google states that putting noindex inside robots.txt itself is not supported.

Removal is not instant. Google has to recrawl the page to see a newly added rule, and its documentation notes that for less important pages that can take months. The URL Inspection tool shows the HTML Googlebot received, and the Page Indexing report lists URLs excluded by noindex.

noindex is also often combined with nofollow (content="noindex, nofollow") or written as none, which Google treats as equivalent to both.

Relation to backlink indexing

A link on a noindexed page cannot be counted as a link from an indexed page, because the page is not in the index. This happens more than link buyers expect: tag archives, paginated author pages, staging copies of guest posts and some profile pages ship with noindex by default.

Submitting such a URL to a backlink indexer will get it crawled, and that crawl is exactly what lets Google read the rule and keep the page out. Checking the source page for a robots rule before submitting avoids spending credits on a page that is designed not to be indexed. The IndexChex backlink monitor records meta robots and X-Robots-Tag values and flags such pages as not indexable.

Related terms

• X-Robots-Tag, the header form of the rule
• robots.txt, which controls crawling, not indexing
• Canonical URL, the preferred tool for duplicates
• rel="nofollow", a link-level hint often confused with the page-level rule

All entries are listed in the glossary.