How to get item url after finding with css selector using selenium

Question

I'm trying to extract a url(link) from a webpage, I used "find_element_by_css_selector" to get the item i want. This item has a url in it. How do I extract this url.

I have tried:

prod_item = browser.find_elements_by_css_selector('div.col-lg-2')
print(prod_item[0].get_attribute('href'))

But I'm getting "None" as output. I would love to use the css_selector because there are many similar items on the page and 'div.col-lg-2' is the attribute that is common to them all. How do solve this problem and get the link?

Here is the full code now:

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.common.exceptions import TimeoutException

url = 'https://auctionmaxx.com/Browse?page=0'

browser = webdriver.Firefox()
browser.get(url)


prod_item = WebDriverWait(browser, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "div.col-lg-2[href]")))

print(prod_item[4].get_attribute('href'))

Answer 1

To print the value of the href attribute you have to induce WebDriverWait for the visibility_of_all_elements_located() and you can use either of the following Locator Strategies:

• Using CSS_SELECTOR:

  browser.get("https://auctionmaxx.com/Browse?page=0")
  prod_item = WebDriverWait(browser, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "div.col-lg-2>div a")))
  print(prod_item[0].get_attribute('href'))
  

• Using CSS_SELECTOR in a single line:

  browser.get("https://auctionmaxx.com/Browse?page=0")
  print(WebDriverWait(browser, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "div.col-lg-2>div a")))[0].get_attribute('href'))
  

• Console Output:

  https://auctionmaxx.com/Listing/Details/321939965/NEW-PUREX-LAUNDRY-DETERGENT-924L
  

• Note : You have to add the following imports :

  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support import expected_conditions as EC