Selenium returns empty values

Question

Can you help me to solve the follow issue? I'm creating a function with Selenium to scrape some jobs from Glassdor, but even though I'm using the correct XPath to get the "salary estimate" it just returns the except results.

xpath = './/span[@class="css-18034rf e1wijj242"]' 

try:
    salary_estimate.append(driver.find_element(By.XPATH, xpath).text)

except:
    salary_estimate.append("#N/A")
    pass

Here is the salary estimate on the job card and the search link:

salary estimate position

Answer 1

That because the locator you are using is dynamically generated and changes on every reload and will not return anything

Try using this

//span[@data-test='detailSalary']

This will return you all ranges and when you do a .text you will get output as $74T - $1L (Glassdoor Est.)

Then to further Filter out the Salary ranges if you want , you have to extract just the parent text element

 parent_element = driver.find_element(By.XPATH,"//span[@data-test='detailSalary']");
 print(driver.execute_script('return arguments[0].childNodes[0].textContent;', parent_element).strip())

This will give output as $55.00 - $85.00 Per hour

Similarly you can extract all by iterating through the list of elements

parent_list = driver.find_elements(By.XPATH,"//span[@data-test='detailSalary']");


for el in parent_list:
  
   print(driver.execute_script('return arguments[0].childNodes[0].textContent;', el).strip());