stop unwanted traffic on the website

Question

I have unwanted traffic on my website. for example.

1. Someone make a html or js file and open a webpage of my website.
2. Spam bots, which browse my website.

This type of traffic dont give me any benefit rather it is using bandwidth.

I want to know how to detect this type of traffic on my website and stop them.

If i come to know the ip address or referer url then i block that ip or referer url from htaccess. but in that case i have add ip or referer url each time. i want it automatic. something like it detect or identify the spam hits and block them.

Answer 1

Please add robots.txt to your web site root to prevent most automatic crawlers from accessing you site.

You could also come up with a rule that redirects access to error page if Referrer is set (client comes through a link, search engine or such). This only allows direct access to the web site.

The real answer: do not bother

• Trying to limit unwanted access will most likely just hurt legitimate visitors.
• Setting up the limitations will cost more in time than the saved bandwidth is worth
• If you bandwidth is really so expensive, change service provider