Search code examples
webdnsseosearch-engine

Blocking Google (and other search engines) from crawling domain

We want to open a new domain for certain purposes (call them PR). The thing is we want the domain to point to the same website we currently have.

We do not want this new domain to appear on search engines (specifically Google) at all.

Options we've ruled out:

  • Robots.txt can't be used - it will work the same on both domains, which isn't what we want.
  • The rel=canonical doesn't block - only suggests to index a similar page instead. The original page might end up being indexed.

Is there a way to handle this?

EDIT

Regarding .htaccess suggestions: we're on IIS7.

Solution

  • rel=canonical is not a suggestion. It tells Google exactly which page to use.

    Having said that, when serving pages that are in the domain you do not want indexed you can use the `x-robots-tag- to block those pages from being indexed:

    Simply add any supported META tag to a new X-Robots-Tag directive in the HTTP Header used to serve the file.

    Don't include this document in the Google search results:

    X-Robots-Tag: noindex