Search code examples
seoindexinghttp-status-code-404search-enginegoogle-crawlers

prevent google to index dynamic error pages (none 404)

There are some none 404 error pages on my website. what is the best way to stop google from indexing them?

option 1

header("HTTP/1.0 410 Gone");

what if the content is not gone? for example: the article does not exist. or wrong parameter has been caught

option 2

<meta name="robots" content="noindex" />

does it only affect one page or the whole domain?

option 3

using 404 which will make some other problems and I would like to avoid.

robot.txt

this option will not work since the error will depend on the database and is not static.

Solution

  • Best practice is to do a 301 redirect to similar content on your site if content is removed.

    To stop Google indexing certain areas of your site use robots.txt

    UPDATE: If you send a 200 OK and add the robots meta tag (Option 2 in your question) - this should do what you want.