What are the max. page limits a Search Engine can crawl?

by Geethalakshmi 2010-07-18 12:59:35

What are the max. page limits a Search Engine can crawl?


One of the most asked question in forums and in message boards what is the maximum depth a search engine can crawl for a page. What is the maximum size Of html /pdf that can indexed by top search engines.

1. For google: As per our latest research Google has a maximum crawl and cache depth of 1 MB only ( excluding images / graphics ) . It Used to be 100 kb then they increased to 250 kb then to 500kb and the latest update is 1 MB per file.

2. Yahoo overtakes Google by a long way, Their indexing and caching limit is 5 MB

3. MSN Search engine: Its very unpredictable for MSN but from our experiment MSN can cache upto 3 MB, we never tested about that probably someone in their search quality team can answer that.


Tagged in:

841
like
0
dislike
0
mail
flag

You must LOGIN to add comments