Today i have one question that how google crawler find each and every page over the internet during crawling please share me the answers if you have.
Review the below link if you need any more on this.
Thanks in advance !!!!
It happens that crawling is process of storing cached data, in cached data, the search engine looks for hyperlinks, may it be textual or image hyperlinks. Then after it has found one, it opens that page to cache and starts finding for links in it. The process goes on until there are no more links to be found.
So, in the this long chain of hyperlinks, it is almost certain that most of the internet is included. But, that doesn't mean that it has crawled everything. Many new websites are left out in this, because of lack of links to it. and also some websites are not crawled because they are not intended to be discovered.