Search code examples
pythonlinuxselenium-webdrivercpanel

Python / Selenium-webdrivers doesn't work on cloudlinux x86 64bit / cpanel


My solution works locally on windows 10. with python 3.12

Deployed to a linux server with Python 3.11.5, I get the exception:

Service /home/project/.cache/selenium/chromedriver/linux64/119.0.6045.105/chromedriver unexpectedly exited. Status code was: -5
Traceback (most recent call last):
  File "/home/project/repositories/fam/fam.py", line 23, in init_browser
    browser = webdriver.Chrome(options)
              ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/project/virtualenv/repositories/fam/3.11/lib/python3.11/site-packages/selenium/webdriver/chrome/webdriver.py", line 45, in __init__
    super().__init__(
  File "/home/project/virtualenv/repositories/fam/3.11/lib/python3.11/site-packages/selenium/webdriver/chromium/webdriver.py", line 53, in __init__
    self.service.start()
  File "/home/project/virtualenv/repositories/fam/3.11/lib/python3.11/site-packages/selenium/webdriver/common/service.py", line 102, in start
    self.assert_process_still_running()
  File "/home/project/virtualenv/repositories/fam/3.11/lib/python3.11/site-packages/selenium/webdriver/common/service.py", line 115, in assert_process_still_running
    raise WebDriverException(f"Service {self._path} unexpectedly exited. Status code was: {return_code}")
selenium.common.exceptions.WebDriverException: Message: Service /home/project/.cache/selenium/chromedriver/linux64/119.0.6045.105/chromedriver unexpectedly exited. Status code was: -5

Python code:

def init_browser():
    #Init browser
    logging.info("Initializing browser object")
    try:
        options = webdriver.ChromeOptions()
        options.add_argument('--headless=new') 
        # block image loading
        options.experimental_options['prefs'] = {
        "profile.managed_default_content_settings.images": 2
        }
        browser = webdriver.Chrome(options=options)
    except Exception as ex:
        logging.exception(ex)
        sys.exit()
    else:
        return browser

Relevant log output

shell
Selenium Manager binary found at: /home/project/virtualenv/repositories/fam/3.11/lib/python3.11/site-packages/selenium/webdriver/common/linux/selenium-manager
Executing process: /home/project/virtualenv/repositories/fam/3.11/lib/python3.11/site-packages/selenium/webdriver/common/linux/selenium-manager --browser chrome --debug --output json
chromedriver not found in PATH
chrome not found in PATH
chrome not found in the system
Discovering versions from https://googlechromelabs.github.io/chrome-for-testing/last-known-good-versions-with-downloads.json
Required browser: chrome 119.0.6045.105
chrome 119.0.6045.105 already exists
chrome 119.0.6045.105 is available at /home/project/.cache/selenium/chrome/linux64/119.0.6045.105/chrome
Discovering versions from https://googlechromelabs.github.io/chrome-for-testing/known-good-versions-with-downloads.json
Required driver: chromedriver 119.0.6045.105
chromedriver 119.0.6045.105 already in the cache
Driver path: /home/project/.cache/selenium/chromedriver/linux64/119.0.6045.105/chromedriver
Browser path: /home/project/.cache/selenium/chrome/linux64/119.0.6045.105/chrome
Using driver at: /home/qproject/.cache/selenium/chromedriver/linux64/119.0.6045.105/chromedriver
Started executable: `/home/project/.cache/selenium/chromedriver/linux64/119.0.6045.105/chromedriver` in a child process with pid: 365909

I've submitted a bug at github SeleniumHQ. They gave up. Hope you guys can help! https://github.com/SeleniumHQ/selenium/issues/13223

What have I tried?

I've tried altering the code to:

  • use local binarys for webdriver and browser executable
  • Both chrome and Firefox
  • use custom port numbers like 8080 and 80 with
--remote-debugging-port=8080

added arguments:

-disable-dev-shm-usage
--no-sandbox

and ANY combination of the above

Linux filepermissions checked. You can see them at the github link.

The only new error I have is when I'm using Chrome with binarys I downloaded myself:

2023-12-01 13:19:23 - INFO - root - Initializing browser object
2023-12-01 13:19:23 - INFO - root - Browser type is chrome
2023-12-01 13:19:23 - INFO - root - Local files are used
2023-12-01 13:19:23 - DEBUG - selenium.webdriver.common.service - Started executable: `/home/project/home/selenium_drivers/chromedriver-linux64/chromedriver` in a child process with pid: 1431566
2023-12-01 13:19:24 - ERROR - root - Message: session not created: Chrome failed to start: exited normally.
  (session not created: DevToolsActivePort file doesn't exist)
  (The process started from chrome location /home/project/home/selenium_bin/chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)

Solution

  • A2 shared hosting does not support "anything that runs chrome or firefox"

    So, after many hours of grief. I decided to contact my hosting support. After they spent 1 hour of chatting with me and trying different solutions, they turned around and said:

    [16:35:21]  [name]: Unfortunately, we don't support puppet on shared hosting.
    [16:35:35]  [name]: puppet/selenium/anything that runs chrome or firefox is not supported in shared hosting.
    

    This is not in their terms og knowledge base. You heard it here first.

    A2 shared hosting does not support "anything that runs chrome or firefox"