middlewares.py in scrapy is not executed as expected while trying to use multiple user-agents

Question

I am trying to use multiple agents in my scrapy project. I found this script for middlewares.py here:

import random
from scrapy.conf import settings
from myScrape.settings import USER_AGENT_LIST
import logging

class RandomUserAgentMiddleware(object):

    def process_request(self, request, spider):
        ua  = random.choice(USER_AGENT_LIST)
        print('ua = %s' %ua)
        if ua:
            request.headers.setdefault('User-Agent', ua)
            # check which ua is used
            logging.debug(u'\n>>>>> User-Agent: %s\n' %request.headers)

and the USER_AGENT_LIST is in the settings.py:

USER_AGENT_LIST = [
    'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:43.0) Gecko/20100101 Firefox/43.0',
    'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.7 (KHTML, like Gecko) \
        Chrome/16.0.912.36 Safari/535.7',
    'Mozilla/5.0 (Windows NT 6.2; Win64; x64; rv:16.0) Gecko/16.0 Firefox/16.0',
    'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_3) AppleWebKit/534.55.3 \
        (KHTML, like Gecko) Version/5.1.3 Safari/534.53.10'
]

DOWNLOADER_MIDDLEWARES = {
    'myScrape.middlewares.RandomUserAgentMiddleware': 400,
    'scrapy.downloadermiddleware.useragent.UserAgentMiddleware': None,
    # Disable compression middleware, so the actual HTML pages are cached
}

But it does not work as I expect. I still see the Scrapy user-agent while crawling. The print function in middlewares.py is called and shows the correct ua, but the logging output gives the Scrapy agent.

How does it work? Do I need to call it somehow from my spider script?

Answer 1

As pointed out by eLRuLL, it was a typo. I was missing the 's' on downloadermiddlewares for the correct path of the UserAgentMiddleware