Search code examples
pythonseleniumselenium-webdriverweb-scrapingwebdriver

Getting a list of elements by using Selenium


I am attempting to obtain the differents sports that are highlighted on a bookmaker's betting page in order to extract later on the different odds and compare them with other bookmaker's pages. The inspect result is shown on the picture. enter image description here The code I tried is the following :

sport_list = driver.find_elements(By.CLASS_NAME, "filters__content")

for sport in sport_list:

print(sport.text)

Unfortunately sport_list is only composed of one element why I would expect it to have the same number of elements than the number of sports visible in the inspect section.

It loops only 1 time and returns the following : " Soccer\nTennis\nBasketball\nAmericanFootball ..."

Can someone help me figure out what is going wrong ?

Thanks in advance.

Brieuc


Solution

  • What happens?

    Cause you are selecting the <div> that contains the list you get an resultset with one element (<div>) only.

    driver.find_elements(By.CLASS_NAME, "filters__content")
    

    How to fix?

    Selecting the <li>s in <div> and get its text use xpath or as in following example css selectors.

    for e in driver.find_elements(By.CSS_SELECTOR,'.filters__content li'):
        print(e.text)
    

    Output

    Football
    Tennis
    Basketball
    Football Américain
    Hockey sur glace
    Esports
    Volleyball
    Handball
    Boxe
    Rugby
    Snooker
    Fléchettes
    Cyclisme
    Waterpolo
    MMA
    Stock Car Racing