Search code examples
php.htaccessgooglebot

Block file_get_content blocks the google bot?

I see this; how to block curl or file_get_contents, page of this site. The code is for .htaccess for blocking the file_get_contents of php.

RewriteEngine  On 
RewriteCond%  {HTTP_USER_AGENT}  ^ $ [OR] 
RewriteCond%  {HTTP_ACCEPT}  ^ $
RewriteRule  ^  -  [L, F]

My question is What is the possibility of blocking the google bot too?

Solution

  • You can test it out by going into your Google Webmaster Tools account and fetching a page on your site as Googlebot. That would be a definitive test.

    But it looks like that code is blocking user agents that have no value. Google always identifies itself as Googlebot. It also provides a value for the v HTTP header. So I think you'd be fine.