Search code examples
pythonhtmlcssweb-scrapingscrapy

How to get the value of <data ..> with css selector with Scrapy?


This is the html element on the website :

<data value="5200" class="text-red-600 font-bold font-arabic  undefined"><span class="mr-1">5</span><span class="mr-1">200</span><span class="text-xs font-medium">DT</span></data>

I want to get the value "5200".

Here's what I tried :

 bike.css('data::text').get() 
bike.css('data::value').get()
bike.css('data').get()

The last one gets all the <data> element not the value.

Here's my code :

class BikespiderSpider(scrapy.Spider):
    name = "bikespider"
    allowed_domains = ["Domain_here"]
    start_urls = ["Link_here"]

    def parse(self, response):
        
        bikes = response.css('article.mx-0')

        for bike in bikes:
            yield {
                'name' : bike.css('h2::text').get(),
                'price' : bike.css ()
            }




            # bikes = response.css("h2::text").getall()

Solution

  • bike.css('data::attr(value)').get() as documented at https://docs.scrapy.org/en/latest/topics/selectors.html#extensions-to-css-selectors