It is well-known that, in the turn of the century, Google stood out of other search engines because of its revolutionary PageRank algorithm (despite the fact that mathematically, PageRank is just a very simple application of Perron-Frobenius theorem). However, very little information about those search algorithms used by pre-Google search engines can be found on the web.
Does anybody know how big names like Lycos, Alta Vista, Excite, Yahoo or Ask Jeeves perform web search and page ranking? Is there any material that documents those algorithms in more details?
For instance, I learn from this short history of early search engines that Excite did "use statistical analysis of word relationships to improve relevancy of searches on the Internet", while Yahoo maintained "a highly regarded directory of sites that were cataloged by human editors". That's a valuable piece of information, but the descriptions are still too vague.
There is this documentary about the history of the internet (up to 2008) called Download: The History of the Internet. I think it mentioned that yahoo curated and categorized popular websites by hand.
I think it worked like an address book, the websites needed to opt-in and provide yahoo with it's web address and category.