There are some bots that just keep hitting my site over and over again.
http://proxy.parser.by/proxy.php (Referrer)
I don't want to even reply to anyone that is requesting a .php, or .htm, or .html file.
What is the best way of not responding to such requests?
Update: (I don't want to incur the cost of responding)
This is probably best done at the server level, before the request even gets to Django. For example, in Apache, you can use mod_rewrite
for access control. This rule rejects all requests with paths ending with .php
, .htm
, or .html
:
RewriteRule \.(php|html?)$ - [F]
The Apache documentation explains how to block requests by user agent, by referer, by orginating IP address, and so on.