Search code examples
phpmysqlblockingweb-crawler

Block IP addresses in .htaccess or in PHP from database?

I am building a bot-trap / crawler-trap to my website:

There is a hidden link somewhere on the page, which normal users do not see, but a robot sees. The link is also indicated in robots.txt, so Google will not fall into the trap. When a bot opens the hidden page, the IP automatically gets red-flagged in MySQL.

My question is:

  1. When a new IP gets flagged, which way is better to block that: Regenerate the .htaccess file, with the new IP added to it, so the webserver itself is going to do the blocking.
  2. Do not mess with .htaccess, rather look up the IP table from MySQL every time someone loads the page, and then decide in PHP what to do with the user.
Solution

  • I would definitely go with option 2. The only reason being that I would be very uncomfortable that I have a .htaccess file being played with at random intervals in the website all the time. It is a nagging feeling like having a gun on my forehead all the time.

    If it is db driven the worst that can happen in a screw up is that some black listed IP got access still. With htaccess if there is a screwup, not only does every user's experience get messed up, secure data can be compromised as well..