Search code examples
wordpressregistrationbotsgoogle-crawlers

How to allow bots/crawlers and only registered users in wordpress?


How I can allow bots to crawl my website and close it for all visitors? When visitor comes on any url from google he should see registration form, and google bot should see everything. Can I do this?

I found this plugin: http://wordpress.org/plugins/registered-users-only/

Can I now allow google bot by robots.txt and how?


Solution

  • That's against Google's TOS, but you can ask the user to register when he visits a second page on your website.

    You can read more here about First Click Free.

    But if you really want to block people and allow bots, you will need either a plugin or a custom made script that filters the user agents.