Search code examples
pythonselenium-webdriverxpathwebdriver

Unable to click an element and scrape web using xpath and id- selenium


I have been trying to create a code to select an element on a web page- https://tmrsearch.ipindia.gov.in/eregister/ using xpath and element id with the help of Selenium library. I have mentioned two versions of code, first one is using id-

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.action_chains import ActionChains
from bs4 import BeautifulSoup
import pyautogui
import time

url = "https://tmrsearch.ipindia.gov.in/eregister/"
browser = webdriver.Chrome()
browser.get(url)

# Allow the page to load completely
time.sleep(5)

# Wait for the element to be present
mouse_tracker = WebDriverWait(browser, 10).until(
    EC.presence_of_element_located((By.ID, "btnviewdetails"))
)

# Move the mouse to the element
ActionChains(browser).move_to_element(mouse_tracker).perform()

# Wait for the element to be visible
mouse_tracker = WebDriverWait(browser, 10).until(
    EC.visibility_of_element_located((By.ID, "btnviewdetails"))
)

# Perform any further actions
time.sleep(10)

In this case the error is coming as follow-

ERROR***

DevTools listening on ws://127.0.0.1:65022/devtools/browser/1a356b85-193a-44c6-89df-e1a4441e2b2b
Created TensorFlow Lite XNNPACK delegate for CPU.
Traceback (most recent call last):
  File "c:\Users\admin\Desktop\Web scraping\app5.py", line 18, in <module>
    mouse_tracker = WebDriverWait(browser, 10).until(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\admin\AppData\Roaming\Python\Python312\site-packages\selenium\webdriver\support\wait.py", line 105, in until  
    raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:
Stacktrace:
        GetHandleVerifier [0x00007FF6CCB83E52+31618]
        (No symbol) [0x00007FF6CCAFB0B9]
        (No symbol) [0x00007FF6CC9B888A]
        (No symbol) [0x00007FF6CCA08524]
        (No symbol) [0x00007FF6CCA0862C]
        (No symbol) [0x00007FF6CCA4F787]
        (No symbol) [0x00007FF6CCA2D14F]
        (No symbol) [0x00007FF6CCA4CA80]
        (No symbol) [0x00007FF6CCA2CEB3]
        (No symbol) [0x00007FF6CC9FA46B]
        (No symbol) [0x00007FF6CC9FB001]
        GetHandleVerifier [0x00007FF6CCE8A02D+3202397]
        GetHandleVerifier [0x00007FF6CCED6A4D+3516285]
        GetHandleVerifier [0x00007FF6CCECC4C0+3473904]
        GetHandleVerifier [0x00007FF6CCC35D56+760454]
        (No symbol) [0x00007FF6CCB06B5F]
        (No symbol) [0x00007FF6CCB01CF4]
        (No symbol) [0x00007FF6CCB01E82]
        (No symbol) [0x00007FF6CCAF122F]
        BaseThreadInitThunk [0x00007FFFCFA17344+20]
        RtlUserThreadStart [0x00007FFFD09BCC91+33]

The other version of code which is using Xpath is as follow-

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By

# Specify the path to the ChromeDriver executable
service = Service("C:/chromedriver.exe")

# Create Chrome options
options = Options()
options.add_argument("--disable-extensions")
options.add_argument("--disable-infobars")

# Create the Chrome driver with the options
driver = webdriver.Chrome(service=service, options=options)

# Open the URL
driver.get("https://tmrsearch.ipindia.gov.in/eregister/")

# Use the updated method to find the element by XPath
element = driver.find_element(By.XPATH, "/html/body/form/table/tbody/tr[1]/td/a")

# Print out the element to verify it was found
print(element)

# You can interact with the element if needed, for example:
# element.click()

# Close the browser
driver.quit()

In this case, the error is coming as follow-

**ERROR

DevTools listening on ws://127.0.0.1:63892/devtools/browser/b293b9af-7147-4650-8f02-fb096fc509fa
Traceback (most recent call last):
  File "c:\Users\admin\Desktop\Web scraping\app6.py", line 21, in <module>
    element = driver.find_element(By.XPATH, "/html/body/form/table/tbody/tr[1]/td/a")
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\admin\AppData\Roaming\Python\Python312\site-packages\selenium\webdriver\remote\webdriver.py", line 741, in find_element
    return self.execute(Command.FIND_ELEMENT, {"using": by, "value": value})["value"]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\admin\AppData\Roaming\Python\Python312\site-packages\selenium\webdriver\remote\webdriver.py", line 347, in execute
    self.error_handler.check_response(response)
  File "C:\Users\admin\AppData\Roaming\Python\Python312\site-packages\selenium\webdriver\remote\errorhandler.py", line 229, in 
check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/form/table/tbody/tr[1]/td/a"}
  (Session info: chrome=126.0.6478.61); For documentation on this error, please visit: https://www.selenium.dev/documentation/webdriver/troubleshooting/errors#no-such-element-exception
Stacktrace:
        GetHandleVerifier [0x00007FF7D8813E52+31618]
        (No symbol) [0x00007FF7D878B0B9]
        (No symbol) [0x00007FF7D864888A]
        (No symbol) [0x00007FF7D8698524]
        (No symbol) [0x00007FF7D869862C]
        (No symbol) [0x00007FF7D86DF787]
        (No symbol) [0x00007FF7D86BD14F]
        (No symbol) [0x00007FF7D86DCA80]
        (No symbol) [0x00007FF7D86BCEB3]
        (No symbol) [0x00007FF7D868A46B]
        (No symbol) [0x00007FF7D868B001]
        GetHandleVerifier [0x00007FF7D8B1A02D+3202397]
        GetHandleVerifier [0x00007FF7D8B66A4D+3516285]
        GetHandleVerifier [0x00007FF7D8B5C4C0+3473904]
        GetHandleVerifier [0x00007FF7D88C5D56+760454]
        (No symbol) [0x00007FF7D8796B5F]
        (No symbol) [0x00007FF7D8791CF4]
        (No symbol) [0x00007FF7D8791E82]
        (No symbol) [0x00007FF7D878122F]
        BaseThreadInitThunk [0x00007FFFCFA17344+20]
        RtlUserThreadStart [0x00007FFFD09BCC91+33]

This is the website- https://tmrsearch.ipindia.gov.in/eregister/ I want to select the first button with the text Trade Mark Application/Registered Mark

I am attaching the screenshot of the page as well. enter image description here

I even also want to remove the info bar appearing above- "Chrome is being controlled by automated software". please help me out in resolving the error.


Solution

  • As mentioned in the other answer, root cause of the issue is presence of a FRAME which has surrounded the targeted element.

    Here is the full working code with selenium waits to handle it effectively:

    import time
    from selenium import webdriver
    from selenium.webdriver.common.by import By
    from selenium.webdriver.support.ui import WebDriverWait
    from selenium.webdriver.support import expected_conditions as EC
    
    driver=webdriver.Chrome()
    driver.maximize_window()
    driver.get("https://tmrsearch.ipindia.gov.in/eregister/")
    wait = WebDriverWait(driver, 10)
    
    # switch into the frame context
    wait.until(EC.frame_to_be_available_and_switch_to_it((By.NAME, "eregoptions")))
    
    # click on the targeted element
    wait.until(EC.element_to_be_clickable((By.ID, "btnviewdetails"))).click()
    
    # come out of frame
    driver.switch_to.default_content()
    time.sleep(10)