Cabrillo College Library

Search Engines -- How Well Do they Cover the Internet??

 
Consider what's inside the black rectangle to be the Internet.  Consider each colored shape inside to be a search engine. 

Each search engine covers only part of the Internet. 

Major search engines have frequently claimed to keep up with the growth of the Internet. It's been assumed that search engines index roughly the same materials and cover a relatively large proportion of the Web. 

Studies published in 1998 and 1999 indicate that none of the major search engines cover all of the Internet, and that the overlap in coverage is relatively low. 


 

1998 study  (1) 
Search engine     Coverage 
 
HotBot 58%
AltaVista 47%
Northern Light 33%
Excite 23%
Infoseek 17%
Lycos 4%
 
1999 study  (2) 
Search engine     Coverage 
 
Northern Light 38%
AltaVista 37%
HotBot 27%
Google! 19%
Infoseek 19%
Excite 14%
 

If you compare the search engines for searches on quite specific topics, you can show that there is not that much overlap.


 

__________________ 
(1) Steve Lawrence and C. Lee Giles, "Searching the World Wide Web," Science 280 (3 April 1998):98-100. 
(2) Steve Lawrence and C. Lee Giles, "Accessibility of Information on the Web," Nature 400 (8 July 1999):107-109. 
 

tns 7/99; rev. 2.01