I don't know why, but there's been an error recently:
File "C:\Users\name\PycharmProjects\splash\project\project\spiders\scrapy.py", line 5, in <module>
class ScrapySpider(scrapy.Spider):
AttributeError: 'module' object has no attribute 'Spider'
My full code:
import scrapy
from scrapy_splash import SplashRequest
class Spider(scrapy.Spider):
name = "spide"
start_urls = [
many links
]
def start_requests(self):
for url in self.start_urls:
yield SplashRequest(url, self.parse,
endpoint='render.html',
args={'wait': 0.5},
)
def parse(self, response):
"my parsed info"
What's the problem and how can it be solved?
P.S. I set up ScrapySplash through this tutorial
The problem is in the name of your python file:
C:\Users\name\PycharmProjects\splash\project\project\spiders\scrapy.py
When you run
import scrapy
Python is trying to import the current file (that you've written your code in) as the scrapy module. To solve this, rename your file to something other than scrapy.py
. This will mean that when Python tries to import scrapy
, it will import the scrapy.py
file in your site-packages
folder, assuming you've installed scrapy correctly (pip install scrapy
).