I'm learning Python and how to use Scrapy. I've been scraping "http://quotes.toscrape.com/" and everything worked well until I tried to organized data using items. It seems the error is
from ..items import QuotetutorialItem
ImportError: attempted relative import with no known parent package"
I have not been able to find the proper way to import items so your help would be highly appreciated!
Here is the code of my spiders:
import scrapy
from ..items import QuotetutorialItem
class QuoteSpider(scrapy.Spider):
name = "quotes"
start_urls = ["http://quotes.toscrape.com/"]
def parse(self, response):
items = QuotetutorialItem()
all_div_quotes = response.css("div.quote")
for quotes in all_div_quotes:
title = quotes.css("span.text::text").extract()
author = quotes.css(".author::text").extract()
tag = quotes.css(".tag::text").extract()
items["title"] = title
items["author"] = author
items["tag"] = tag
yield items
The code of items.py:
import scrapy
class QuotetutorialItem(scrapy.Item):
# define the fields for your item here like:
title = scrapy.Field()
author = scrapy.Field()
tag = scrapy.Field()
And here the structure:
Why I generally advise is to append your project's path to PYTHONPATH
:
export PYTHONPATH="${PYTHONPATH}:/path/to/quotetutorial/"
and inside your quotes_spider.py
use:
from quotetutorial.items import QuotetutorialItem
Now if you want to import package in items.py
from say quotes_spider.py
, the following should do the trick:
from quotetutorial.spiders.quotes_spider import QuotetutorialItem
Alternatively, the following will also work:
export PYTHONPATH="${PYTHONPATH}:/path/to/quotetutorial/quotetutorial/"
and then:
from items import QuotetutorialItem
or
from spiders.quotes_spider import QuotetutorialItem