Search code examples
pythonseleniumxpathcss-selectorswebdriverwait

Python - Tor Browser through Firefox, unable to click button

I have been trying to access a certain site (dumpert.nl) through Tor Browser as proxy via Firefox. The reason I am using Tor Browser is so I can enter the website with a different IP address every time I enter the website. I know this is possible but I have not yet found the way to do this. I have found multiple ways to do this, but they have not (yet) worked for me. Help is wanted on this part as well.

The real problem is I am having trouble with the Accept Cookie page of this website. When I manually click the button to accept the cookies nothing happens. I can't progress to the next page. If i use the .click() function of Selenium nothing happens either, the page is fully loaded so this is not the issue. The buttons do not work for some reason and I have no clue why. I don't know if it's an Tor problem or using Firefox via Tor problem.

I use the following code to navigate to the website:

    from selenium import webdriver
    from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
    import os
    
    torexe = os.popen(r'C:\Users\nick\Desktop\Tor Browser\Browser\TorBrowser\Tor\tor.exe')
    profile = FirefoxProfile(r'C:\Users\nick\Desktop\Tor Browser\Browser\TorBrowser\Data\Browser\profile.default')
    profile.set_preference('network.proxy.type', 1)
    profile.set_preference('network.proxy.socks', '127.0.0.1')
    profile.set_preference('network.proxy.socks_port', 9050)
    profile.set_preference("network.proxy.socks_remote_dns", False)
    profile.update_preferences()
    driver = webdriver.Firefox(firefox_profile= profile, executable_path=r'C:\Webdrivers\geckodriver.exe')
    driver.get("http://dumpert.nl")
    driver.find_element_by_xpath("/html/body/div[2]/div/div[2]/a").click() #cookie click

    #Rest of my code doing stuff not important for this issue
Solution

  • To open the webpage http://dumpert.nl and click() on the desired button you have to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategies:

    • Using CSS_SELECTOR:

      from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
import os

torexe = os.popen(r'C:\Users\Soma Bhattacharjee\Desktop\Tor Browser\Browser\TorBrowser\Tor\tor.exe')
profile = FirefoxProfile(r'C:\Users\Soma Bhattacharjee\Desktop\Tor Browser\Browser\TorBrowser\Data\Browser\profile.default')
profile.set_preference('network.proxy.type', 1)
profile.set_preference('network.proxy.socks', '127.0.0.1')
profile.set_preference('network.proxy.socks_port', 9050)
profile.set_preference("network.proxy.socks_remote_dns", False)
profile.update_preferences()
driver = webdriver.Firefox(firefox_profile= profile, executable_path=r'C:\WebDrivers\geckodriver.exe')
driver.get("http://dumpert.nl")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a.approve-btn[title^='And yes']>span"))).click()

    • Using XPATH:

      from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
import os

torexe = os.popen(r'C:\Users\Soma Bhattacharjee\Desktop\Tor Browser\Browser\TorBrowser\Tor\tor.exe')
profile = FirefoxProfile(r'C:\Users\Soma Bhattacharjee\Desktop\Tor Browser\Browser\TorBrowser\Data\Browser\profile.default')
profile.set_preference('network.proxy.type', 1)
profile.set_preference('network.proxy.socks', '127.0.0.1')
profile.set_preference('network.proxy.socks_port', 9050)
profile.set_preference("network.proxy.socks_remote_dns", False)
profile.update_preferences()
driver = webdriver.Firefox(firefox_profile= profile, executable_path=r'C:\WebDrivers\geckodriver.exe')
driver.get("http://dumpert.nl")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[@class='approve-btn']/span[starts-with(., 'Yes')]"))).click()

    • Browser Snapshot:

    tor_page