Search code examples
pythonseleniumweb-scrapingcss-selectorswebdriver

Finding elements by class name inside a loop, selenium

I need to scrape in python a page that has some classes that are number like class1, class2, class3, ... classN and I would like to locate the text contained in all those classes.

I tried using find_element() function:


list = []
for i in range(N-1)
  list.append(driver.find_element(By.CLASS_NAME, "class" + str(i+1) ).text))

but unfortunately I get this error message:

raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":".class1"}
  (Session info: chrome=107.0.5304.68)

Is there any other way I could succeed in this scraping?

EDIT

This is the page I would like to scrape and I would like to get the games won by both players per every set.

In the source code the classes are named "smh__part smh__home smh__part--" + str(i+1) and respectively "away"

Solution

  • Class_name only accept single class name not multiple. Instead use css selector. use python format() function

    list = []
for i in range(N-1):
  list.append(driver.find_element(By.CSS_SELECTOR, "[class='smh__part  smh__home smh__part--{}']".format(i+1)).text)

    OR CLASS_NAME with single class value

    list = []
for i in range(N-1):
  list.append(driver.find_element(By.CLASS_NAME, "smh__part--" + str(i+1) ).text)