The directly pasted XPATH of a link into selenium is leading to another link

Question

I am automating the login and profile completion of a Linkden profile. I have been able to get to the home page and past the login page. I am trying to go to the profile page by simulating the click of the link. In order to do this I used the XPATH I directly copied from the inspect page but instead of going to the profile page it is going to a different page. I am not doing this by id as I wish for this code to work for anyone who plugs their login information in.

Here is the code below:

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
import time
import json


with open('information.json', 'r') as file:
    bio_data = json.load(file)


username = ''
password = ''
options = webdriver.ChromeOptions()
# options.add_argument('--headless')  # for headless mode
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()))

# Login -------------------------------------------------
try:
    driver.get('https://www.linkedin.com/login')
    username_field = driver.find_element(By.ID, 'username')
    username_field.send_keys(username)

    password_field = driver.find_element(By.ID, 'password')
    password_field.send_keys(password)

    password_field.send_keys(Keys.RETURN)

    time.sleep(5)

# Posting pages -----------------------------------------

    link = driver.find_element(By.XPATH,'/html/body/div[5]/div[3]/div/div/div[2]/div/div/div/div/div/a')
    link.click()


finally:
    driver.quit()

Answer 1

First a couple comments on your code:

• time.sleep() is inefficient and unreliable, in most cases it's recommended to use WebDriverWait
• Using full path like this is unreadable, unreliable (and ugly) (By.XPATH,'/html/body/div[5]/div[3]/div/div/div[2]/div/div/div/div/div/a'). Searching by class name, id, etc.. is preferred.

Now back to your code, this single line would do what you need. And don't forget to import the right classes. Also time.sleep() in your example can be removed.

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, '//a[@class="ember-view block"]'))).click()