When scraping data from NASDAQ there are tickers like ACHC that have empty pages. ACHC Empty Field
My program iterates through all ticker symbols and when I get to this one it times out because there is no data to grasp. I am trying to figure out a way to check if there is nothing and if so skip the ticker, but continue the loop. The code is pretty long, so Ill post the most relevant part: the beginning of the loop where it opens the page:
## navigate to income statement annualy page
url = url_form.format(symbol, "income-statement")
browser.get(url)
company_xpath = "//h1[contains(text(), 'Company Financials')]"
company = WebDriverWait(browser, 10).until(EC.presence_of_element_located((By.XPATH, company_xpath))).text
annuals_xpath = "//thead/tr[th[1][text() = 'Period Ending:']]/th[position()>=3]"
annuals = get_elements(browser,annuals_xpath)
Selenium doesn't have a built-in method for determining whether an element exists or not, so the most common thing to do is use a try/except block.
from selenium.common.exceptions import TimeoutException
...
try:
company = WebDriverWait(browser, 10).until(EC.presence_of_element_located((By.XPATH, company_xpath))).text
except TimeoutException:
continue
This should keep the loop going without crashing, assuming that continue
works as expected with your loop.