Search code examples
pythonweb-scrapingscrapyweb-crawlerscrapyd

scrapyd error while deploying with scrapyd-client

While executing the command:

scrapyd-deploy default

I'm runnning into an error saying:

File"/home/user/miniconda3/envs/quickcompany/lib/python3.8/site-packages/scrapyd_client/deploy.py", line 23, in <module>
    from scrapy.utils.http import basic_auth_header
ModuleNotFoundError: No module named 'scrapy.utils.http'

I have tried uninstalling and resinstalling the relevant libraries. Also tried using both the github and packaged versions of scrapyd-client.

Solution

  • For those wondering about the answer, there was a glitch in the library. I tried upgrading scrapyd client and it seemed to work fine from there on out