How to locate the LOAD MORE button using Selenium Python

Question

I'm having trouble trying to figure out how to locate the path for a "Load More Button" on this site I am trying to scrape.

https://www.ufc.com/athletes/all

The button html:

<a class="button" href="?gender=All&amp;search=&amp;page=1" title="Load more items" 
 rel="next">Load More</a>  
 "Load More" == $0
 ::after == 0$
 

Answer 1

To click on the element LOAD MORE you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:

• Using CSS_SELECTOR:

  driver.get('https://www.ufc.com/athletes/all')
  WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button#onetrust-accept-btn-handler"))).click()
  WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a[title='Load more items']"))).click()
  

• Using XPATH:

  driver.get('https://www.ufc.com/athletes/all')
  WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[@id='onetrust-accept-btn-handler']"))).click()
  WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[@title='Load more items']"))).click()
  

• Note: You have to add the following imports :

  from selenium.webdriver.support.ui import WebDriverWait
  from selenium.webdriver.common.by import By
  from selenium.webdriver.support import expected_conditions as EC