Search code examples
httpsssl-certificatesearch-engine-bots

how to stop search engines linking to HTTPS?


Background

I manage several websites which have no SSL certificate, hosted on a LAMP shared environment, alongside some that do.

The shared environment has an SSL certificate for all sites, but it's an invalid domain unless connected to with a specific URL that the hosting provider provides.

This creates the uncomfortable environment that any site with no SSL can still be connected to over HTTPS, and be served an invalid certificate (rather than no certificate / no response).

For some reason, google has recently started linking to these non-ssl sites with https:// rather than http:// despite an invalid cert being served. Clicking on this causes a security warning and is probably scaring away 99% of potential clicks.

Question

My question is, given limited server access, what can I do to prevent search engines from linking to an HTTPS version of a website unless there is a valid SSL certificate installed?


Solution

  • Use 301 redirects to accomplish this.

    The search bots don't check SSL cert validity in the first place, so they will connect unlike browsers which check validity and show the warning before a redirect can take place.

    The bot will see the redirect and update the index appropriately given time.

    Example in .htaccess (also works in wordpress):

    <IfModule mod_rewrite.c>
        RewriteEngine On
        RewriteBase /
        RewriteCond %{ENV:HTTPS} on
        RewriteRule (.*) http://%{HTTP_HOST}%{REQUEST_URI} [R=301,L]
    </IfModule>