Search code examples
web-scrapingrecommendation-engine

How google scrape stackoverflow all questions while searching?

I am making search engine. But I want to know, How google scrapes all data of stackoverflow.

As my intuition, Do they save all stackoverflow data in csv file? and when user types some coding question, use some algorithm and recommend users. or anything else,

Or Anything else? Thank you for help.

Solution

  • Storing every data in a csv and running a search will probably cost you hours to retrieve a result.

    Google Search Engine works in 3 stages, Crawling, Indexing, and Serving.

    These algorithms working in these 3 stages are what makes Google so powerful. As they are fine tuned after years of optimization and learning, they can accurate index each webpage and analyze them without just plainly storing everything, just what you suggested.

    Reference: How Search Works