noindex HTML meta tag

The noindex HTML meta tag advises automated Internet bots to avoid indexing a Web page. There are a multitude of reasons why one might want to use this tag. These include preventing robots from trying to index very large database or pages that are very transitory or ones that one wishes to keep slightly more private. Since the burden of honoring a website's noindex tag lies with the author of the search robot, sometimes these tags are willfully ignored. Also the interpretation of the noindex tag is sometimes slightly different from one search engine company to the next.




Bot-specific Directives

There are ways to send the noindex directive only for certain bots, typically by using a different "name" value.

For example, to specifically block Google's bot, specify



or to block Yahoo!'s bot, specify