Scrapy - Use proxy middleware but disable proxy for specific requests

Question

I want to use proxy middleware in my Scrapy but not every request needs a proxy. I don't want to abuse the proxy usage and make the proxy prone to get banned.

Is there a way for me to disable proxy in some requests when the proxy middleware is turned on?

Answer 1

we can add dont_proxy meta and sets it to true on the requests

yield scrapy.Request(
    url, 
    meta={"dont_proxy": True}, 
    callback=self.parse
)