Is it possible to set different settings for different request in the same Scrapy spider?

Question

I want to use Crawlera only for some requests in a Scrapy spider. So I want to set CRAWLERA_ENABLED differently for different requests. Is it possible?

Answer 1

You can use the dont_proxy key in meta for those requests you don't want to use Crawlera. E.g.

# Supposing you have crawlera enabled in `settings.py`
yield scrapy.Request(
    url, 
    meta={"dont_proxy": True}, 
    callback=self.parse
)