Search code examples
search-engineseo

Hide a single content block from search engines?

A hheader.html is included on top of every URL, but its not relevant for search and messing up the all the results beeing the first line of every page... (in the code its the last line but visually its the first, which google is able to notice)

Solution1: You could put the header (content to exculde from google searches) in an iframe with a static url domain.com/header.html and a <meta name="robots" content="noindex" /> ?

  • are there takeoffs?

Solution2: You could deliver it conditionally by apache mod rewrite, php or javascript

  • takeoff(?): google does not like it possibly? will google ever try pages with a standard users's useragent and compare?

  • takeoff: The hidden content will be missing in the google cache version as well...

example: add-header.php:

<?php $path = $_GET['path'];
      echo file_get_contents($_SERVER["DOCUMENT_ROOT"].$path);  ?>

within apache (virtual)host config:

RewriteCond  %{HTTP_USER_AGENT}  !.*spider.*  [NC]
RewriteCond  %{HTTP_USER_AGENT}  !Yahoo.*   [NC]
RewriteCond  %{HTTP_USER_AGENT}  !Bing.*    [NC]
RewriteCond  %{HTTP_USER_AGENT}  !Yandex.*  [NC]
RewriteCond  %{HTTP_USER_AGENT}  !Baidu.*   [NC]
RewriteCond  %{HTTP_USER_AGENT}  !.*bot.*   [NC]
RewriteCond %{SCRIPT_FILENAME} \.htm$ [NC,OR]
RewriteCond %{SCRIPT_FILENAME} \.html$ [NC,OR]
RewriteCond %{SCRIPT_FILENAME} \.php$ [NC]
RewriteRule ^(.*)$  /var/www/add-header.php?path=%1 [L]
Solution

  • Don't send different content to search engines than you would send to a user. ( Your solution 2 ). This is called "cloaking" and is a clear violation of Google's policies. If the difference is significant it will get your site penalized.

    There are better answers anyway. Generally if you want a block to not be indexed include it in a div/iframe that is loaded dynamically after initial page load using javascript - e.g. use ajax.