scrapy shell 'https://www.samsung.com/in/smartphones/galaxy-m/'
fetch('https://searchapi.samsung.com/v6/front/b2c/product/finder/gpv2?type=01010000&siteCode=in&start=1&num=12&sort=onlineavailability&onlyFilterInfoYN=N&keySummaryYN=Y&filter2=03i04')
response.css('product-card-v2__item')
Description: I'm trying to fetch the name, price of products mentioned in the URL. But every time it returns empty list.
The content of that site are dynamic, so you can't access them using xhr. However, there is an api available containing the same stuff you are after. The following is how you can scrape the product names and the categories they belong to from landing page. Feel free to include other relevant fields.
import scrapy
class m52(scrapy.Spider):
name = 'm52Mobi'
start_urls = ['https://searchapi.samsung.com/v6/front/b2c/product/finder/gpv2?type=01010000&siteCode=in&start=1&num=12&sort=onlineavailability&onlyFilterInfoYN=N&keySummaryYN=Y&filter2=03i04']
def parse(self, response):
for item in response.json()['data']['components']:
if not item['name']=='PRODUCT_CAROUSEL':continue
for container in item['parameters']:
cat_name = container['title']
for product in container['products']:
yield {"category":cat_name,"product name":product['name']}