Search code examples
pythonscrapyscrapy-shell

Scrapy Xpath: Extracting @title from img node

I want to extract the @title from the Main Notes According to Your Votes section from this page: https://www.fragrantica.com/perfume/Remy-Latour/Cigar-9351.html

I have fetched the HTML, then tried this line of code on scrapy shell but the output was None:

response.xpath('//*[@id="userMainNotes"]/div/img/@title).extract_first()

What am I doing wrong?

Solution

  • This will work

    response.xpath('//span[contains(@id, "note")]/img[@rel]/@title')

    Do not forget to set USER AGENT to your settings.py

    Crawler data form website use Scrapy 1.5.0 - Python