How to navigate button tag with selenium?

Question

I can't navigate the button using selenium that can automatically click on the "See all people results" by using XPATH for all search results.

# Task 3: Find all people results
people_h2 = driver.find_element(By.XPATH,'//h2[text()="People"]')
parent_div_element = people_h2.find_element(By.XPATH,'..')
ul_element = parent_div_element.find_element(By.XPATH,'following-sibling::ul')
div_element = ul_element.find_element(By.XPATH, 'following-sibling::div')
see_all_people = div_element.find_element(By.CLASS_NAME, "search-results__cluster-bottom-banner")
people_results = see_all_people.find_element(By.TAG_NAME,"a")
people_results.click()
sleep(3)

From the picture above, when I am copying the XPATH for section for me to navigate and click on the button using selenium, the XPATH is changes according to what I am search in the searching bar, where see all people for accountant people and see all people for software engineering people is different. So, I can't used only one path. Do you have any idea for me to use only one coding that can click the see all people results that can be used for any type of people. if use class = search-results__cluster-bottom-banner, it doesn't specifically go to see all people results instead, it will navigate to see all jobs result first.

Answer 1

you can directly go to the same page without even the need of clicking on the "See all people results". And this can be done by looking at the change in the URL upon clicking on that specific button. This means you can use this URL to directly jump to the result page (provided you're already logged in).

import time
from selenium import webdriver
from selenium.webdriver import ChromeOptions, Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.wait import WebDriverWait

options = ChromeOptions()

options.add_argument("--start-maximized")
options.add_experimental_option("useAutomationExtension", False)
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option(
    "prefs",
    {
        "credentials_enable_service": False,
        "profile.password_manager_enabled": False,
        "profile.default_content_setting_values.notifications": 2
        # with 2 should disable/block notifications and 1 to allow
    },
)

driver = webdriver.Chrome(options=options)

url = "https://www.linkedin.com/uas/login"
driver.get(url)
WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.ID,"organic-div")))
container = driver.find_element(By.ID, "organic-div")

# login: fill the email account, password
email = container.find_element(By.ID, 'username')
password = container.find_element(By.ID, 'password')
email.send_keys("xxxxxxxxxxxx")
password.send_keys("xxxxxxxxx")
password.send_keys(Keys.ENTER)
time.sleep(2)
WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.CLASS_NAME, "authentication-outlet")))

key = "accountant people"
driver.get(f"https://www.linkedin.com/search/results/people/?keywords={key}")

WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.CSS_SELECTOR, 'main[id="main"]')))
time.sleep(5)

here are the URLs for LinkedIn searches:

# for all search
search_key = "accountant people"
f"https://www.linkedin.com/search/results/all/?keywords={search_key}"
# for people search
f"https://www.linkedin.com/search/results/people/?keywords={search_key}"
# for job search
f"https://www.linkedin.com/jobs/search?keywords={search_key}"