Search code examples
botsrets

Prevent search bot from triggering download of image from RETS server


I am building a realty website for a client whereby I will be getting all of my data from the RETS server. There is roughly 5,000+ properties in the DB so I pull the images for each property on the fly (as to not overload my server).

Everything is working properly, but I only pull the images when a user views the certain property. So my question is: will a search bot also trigger this image download??

If so, how can I prevent that??


Solution

  • Depends on the search bot. If your images are all served from some known and well-formed URL, then you can tell bots to not hit them by adding a disallow line to your robots.txt file:

    User-agent: *
    Disallow: /images/