Search code examples
c#asp.netbots

prevent all bots from accessing a page

In asp.net, c# web applciation, is it possible to prevent bots from accessing a particular page?

I want to prevent all bots from accessing the payment page on my site.

Solution

  • You can't.

    You can request that the bots don't visit your site (see Projapati's answer for more details on how to do that), or certain directories, or certain files; but it's really just a request. As long as 'Joe Schmoe User' can open a browser and see the page; nothing prevents someone from automating that visit.

    You could try to do something where you block certain IPs that you know are used by various bots/crawlers; but it'd be a losing proposition. You could never say, with any certainty, that it wasn't a bot visiting the page.

    Or you could modify the site so that only authenticated users can access that particular page and hope that nobody goes through the trouble of setting up a fake account to use.