Google's Bots

Google uses several different bots (user-agents). The bot for web search is Googlebot. Google’s other bots follow rules you set up for Googlebot, but you can set up additional rules for these specific bots as well. Blocking Googlebot blocks all bots that begin with "Googlebot".

Here’s a list of Google robots:

* Googlebot: crawls pages from web index and news index
* Googlebot-Mobile: crawls pages for mobile index
* Googlebot-Image: crawls pages for image index
* Mediapartners-Google: crawls pages to determine AdSense content, only crawls sites if show AdSense ads
* Adsbot-Google: crawls to measure AdWords landing page quality, only crawls sites that use Google AdWords to advertise.