Search code examples
web-crawlergoogle-search

Google search and restricted contents?


If a website has a content that will not be seen if someone is not logged in, then, how does Google see it?

The page is an e-commerce, restricted access (a user must log-in to see products) page; as such Google cannot see those products?

What would be the correct strategy to handle this problem?


Solution

  • In case of Google-Adsense this problem might make sense. Google provides site authentication feature for just this: see - Google AdSense launches site authentication

    You can provide google with your UserName & Password and google bot can crawl your protected pages - so that it can serve targeted ads better when a visitor is on that page.

    Also - if this is not the intended purpose then please remember serving different content to Google from what is served to a user without username & password is blatantly wrong.
    Imagine you searched for something on google and as soon as you clicked on the first link - and the website came up with a sign-up page and no useful information!!