Search code examples
pythoncsvselenium-webdriver

How to find dropdown name in HTML code for Selenium's Select class?

I want to download the CSV data shown at the end of the page using Python:

https://www.cboe.com/delayed_quotes/spx/quote_table/

Specifically, before downloading the data, I need Python to select the dropdowns Option Range and Expirations to 'All'

I have the following code to access the website via Python, which works fine.

from selenium import webdriver
from selenium.webdriver.support.ui import Select
from selenium.webdriver.common.by import By


driver = webdriver.Firefox()
driver.get('https://www.cboe.com/delayed_quotes/spx/quote_table')

html = driver.page_source

I want to use the Selenium select function, but I am unable to find the id of the two dropdowns in the HTML code.

element = driver.find_element(By.ID, "????")
Solution

  • First things first, you cannot use Selenium's Select class if the dropdown does not have a <select> or a <option> element. Refer the link to know what you can and what you cannot using Select class.

    Your target HTML does not have a <select> node for the targeted dropdown. Hence you need to interact with the dropdowns in some other way.

    Refer the below working code with explanation in commented lines:

    import time
from selenium import webdriver
from selenium.webdriver import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Firefox()
driver.maximize_window()
wait = WebDriverWait(driver, 10)

driver.get("https://www.cboe.com/delayed_quotes/spx/quote_table")

# Click on I Agree cookies button
wait.until(EC.element_to_be_clickable((By.XPATH, "//button[text()='I agree']"))).click()

# Below 2 lines will send text 'All' to 'Options Range' text box and performs click on key TAB
wait.until(EC.element_to_be_clickable((By.ID, "select_5"))).send_keys("All")
wait.until(EC.element_to_be_clickable((By.ID, "select_5"))).send_keys(Keys.TAB)

# Below 2 lines will send text 'All' to 'Expiration' text box and performs click on key TAB
wait.until(EC.element_to_be_clickable((By.ID, "select_7"))).send_keys("All")
wait.until(EC.element_to_be_clickable((By.ID, "select_7"))).send_keys(Keys.TAB)

# Click on View Chain button
wait.until(EC.element_to_be_clickable((By.XPATH, "//span[text()='View Chain']"))).click()

# Click on download button
wait.until(EC.element_to_be_clickable((By.XPATH, "//a[@download='spx_quotedata.csv']"))).click()
time.sleep(5)