What exactly IS a spider?

A spider is a piece of software that follows links throughout the Internet, grabbing content from sites and adding it to search engine databases.

Spiders follow links from one page to another and from one site to another. That is the primary reason why links to your site are so critical. Getting links to your website from other websites will give the search engine spiders more opportunities to find and re-index your site. The more times they find links to your site, the more times they will stop by and visit. This has been true since spiders began. Recently there has been an incredible amount of attention paid to links. That's because Google came clean and said in public that the number and quality of links to your site will directly impact its rankings in the search results.

AltaVista, AllTheWeb, Teoma, and Google all factor in the number and quality of links to your site when giving your site its ranking.

Spiders find Web pages by following links from other Web pages, but you can also submit your Web pages directly to a search engine and request a visit by their spider. In essence, that is search engine submission. Because so many millions of web masters have submitted their sites over and over, the search engines have responded by putting more emphasis on sites that their spiders find naturally and less and less emphasis on sites submitted directly. So, being found is better than submitting directly.