Search code examples
googlebotgoogle-crawlers

How to prevent bot/googlebot indexing promotional home page?


We have an e-commerce website. Due to some marketing and promotional campaign we are showing app download page/banner/promotion/big image (and nothing else) on our home page if the user is visiting the site for the first time which is cookie based.

But I don't want bots/crawler to see this content(big image) instead they should see the real content which comes after setting up the cookie. URL is same for both the content.

I can clarify more on this. How can I avoid the bots seeing the promotional content?


Solution

  • Well now I am using this function to detect bots/crawlers in php controller code and do the redirection as needed.

    function bot_detected()
    {
      if 
      (
        !isset($_SERVER['HTTP_USER_AGENT'])
        ||
        empty($_SERVER['HTTP_USER_AGENT'])
        ||
        preg_match('/bot|crawl|slurp|spider/i', $_SERVER['HTTP_USER_AGENT'])
        ||    
        preg_match('/scrappy/python/httpclient/Googlebot|DoCoMo|YandexBot|bingbot|ia_archiver|AhrefsBot|Ezooms|GSLFbot|WBSearchBot|Twitterbot|TweetmemeBot|Twikle|PaperLiBot|Wotbox|UnwindFetchor|facebookexternalhit/i', $_SERVER['HTTP_USER_AGENT'])
       ) 
      {
        return TRUE;
      }
      return FALSE;
    }