Search code examples
pythonselenium-webdriverweb-scrapingautomation

selenium with python how to get the result that products available in this category or not

I want to get the number of products in the children categories available or not but I cannot get this. My code is here:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import Select
from selenium.webdriver import ActionChains
from time import sleep
from selenium.webdriver.common.by import By
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.support.ui import WebDriverWait
from selenium.common.exceptions import ElementNotVisibleException


driver = webdriver.Chrome(executable_path='C:\Program Files\JetBrains\PyCharm 2021.1.1/chromedriver.exe')

# Windows Maximize
driver.maximize_window()
# opening website URL
driver.get("http://esindbaad.com/")
# checking catagories one bye one
# Food & Grocery (Beverages)
food = driver.find_element_by_xpath("//*[@id='head-wrapper']/div/div/div/div/ul/li[1]/a")
Beverages = driver.find_element_by_xpath("//*[@id='head-wrapper']/div/div/div/div/ul/li[1]/div/div/div/div/div/div[1]/div[1]/div/ul/li/a")
actions = ActionChains(driver)
actions.move_to_element(food).move_to_element(Beverages).click().perform()
driver.execute_script("window.scroll(0,900)")
# Return back to home page
driver.implicitly_wait(5)
driver.execute_script ( "window.history.go(-1)" )
# Food & Grocery (fruits)
food = driver.find_element_by_xpath("//*[@id='head-wrapper']/div/div/div/div/ul/li[1]/a")
fruits = driver.find_element_by_xpath("//*[@id='head-wrapper']/div/div/div/div/ul/li[1]/div/div/div/div/div/div[2]/div[1]/div/ul/li/a")
actions = ActionChains(driver)
actions.move_to_element(food).move_to_element(fruits).click().perform()
driver.execute_script("window.scroll(0,900)")
# Return back to home page
driver.implicitly_wait(5)
driver.execute_script ( "window.history.go(-1)" )

In this I want to get the result that how many products are available in the Food & Grocery (Beverages) and like other fields.

Solution

  • To get the number of products on a one page you should get the list of them and measure its length.

    Here: 1 I go to the main page 2 Click Food & Grocery 3. Get the products number with a unique locator:

    from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC


driver = webdriver.Chrome(executable_path='/snap/bin/chromium.chromedriver')
driver.implicitly_wait(15)
driver.get('https://esindbaad.com/')
driver.find_element_by_css_selector(".item-vertical.with-sub-menu.hover>a[href*='cat1_id=5&cat2_id=0&cat3_id=0']").click()  # clicking the food & grocery category
wait = WebDriverWait(driver, 30)
wait.until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, ".products-list>.product-layout")))
cards = driver.find_elements_by_css_selector(".products-list>.product-layout")
length = len(cards)
print(length)
driver.quit()

    Get the output: 15. The main solution for your question are these two rows:

    cards = driver.find_elements_by_css_selector(".products-list>.product-layout")
length = len(cards)

    find_elements_by_css_selector is used to find a list of elements. Other then that, you do not have to set same implicit wait many times. Set it only in the beginning and it will be applied.

    Your code has other problems, but I abstract away from them (locators, windows handling etc).