Search code examples
pythonseleniumfor-loopselenium-webdrivercounting

Why wouldn't this star-counting code work?


I tried to count the stars(ratings) of each rating column in this URL 'https://seedly.sg/reviews/p2p-lending/funding-societies'

I use selenium to automate the whole process. but neither stars, star_count, nor star_count_list has any content in it. The code makes logical sense to me and seems fine, may i know what's the problems in my code?

Thanks in advance.

##These are basic setups
from selenium import webdriver 
from selenium.webdriver.common.by import By 
from selenium.webdriver.support.ui import WebDriverWait 
from selenium.webdriver.support import expected_conditions as EC 
from selenium.common.exceptions import TimeoutException
from time import sleep
import pandas as pd

'''Create new instance of Chrome in Incognito mode'''
##Adding the incognito argument to our webdriver
option = webdriver.ChromeOptions()
option.add_argument(" — incognito")
##create a new instance of Chrome
browser = webdriver.Chrome('/Users/w97802/chromedriver')

'''Scrape Basic Info'''
from parsel import Selector
url = 'https://seedly.sg/reviews/p2p-lending/funding-societies'
browser.get(url)
selector = Selector(text=browser.page_source)

####################################################################
##This is the star-count code
'''Count stars simple'''
star_count_list = []

ratingcolumn = browser.find_elements_by_xpath('//div[contains(@class,"qr0ren-7 euifNX")]')
for rows in ratingcolumn:
    star_count = 0
    stars = browser.find_elements_by_xpath('//svg[contains(@stroke,"57CF8D")]')
    for numofstars in range(0,len(stars)):
        star_count += 1
        star_count_list.append(star_count)

print(stars)
print(star_count)
'''Print Stars Result''' 
for i,e in enumerate(star_count_list, start=1):
        print ('\n \n \n ' + str(i) + '. \n', e)  


Solution

  • Though you code id perfect buttThe way you are trying to locate an svg element is not correct.

    You need to replace xpath

    //svg[contains(@stroke,"57CF8D")]
    

    with

    //*[local-name() = 'svg'][contains(@stroke,'57CF8D')]
    

    I'm curious how did you populate such xpath. I've tested in chrome and it doesn't show any finding. Better way to populate in chrome once for surety.

    As an alternative you can use css selector to locate the same element

    div[class='qr0ren-7 euifNX'] svg[stroke='#57CF8D']
    

    in you code:

    stars = browser.find_elements_by_css_selector("div[class='qr0ren-7 euifNX'] svg[stroke='#57CF8D']")