So guys, I just made a fresh installation of Python 3.6 on Centos 7. Created a virtual environment, when I run pip install scrapy, it starts collecting few other dependencies, but fails when it comes to Twisted without returning any error at all. It just freezes and nothing ever happens.
Collecting Twisted>=13.1.0 (from scrapy)
Using cached Twisted-17.9.0.tar.bz2
I've been looking into this error since yesterday, looks like error could be related to Twisted's tar.bz2 extension. I'm new to linux environment and Python, have successfully installed Scrapy on Windows a few months ago using the same pip install scrapy method. But right now, I have no clue how to proceed.
That's the guide I followed to install Python on Centos7. https://www.digitalocean.com/community/tutorials/how-to-install-python-3-and-set-up-a-local-programming-environment-on-centos-7
Could someone please point me in the right direction? Could it really be related to Twisted's compression method? If so, how can I find a way around this issue? Huge thanks in advance.
Couldn't figure out how to get it extract bz2, so ended up installing Twisted and its dependencies manually. For future reference;
Download latest Twisted here
Extract the file
pip install -r top_level.txt. This will install Twisted's dependencies.python setup.py installpip install scrapy.