Running Selenium Webdriver in GitHub Actions for unit tests

Question

I'm trying to run a unit test in GitHub actions before deploying my AWS Lambda, where I'm running Selenium Webdriver with a functioning Selenium-chromium lambda layer (https://github.com/vittorio-nardone/selenium-chromium-lambda). I'm running my environment in Python 3.6 and using ChromeDriver version 95.0.4638.54

I kept on encountering this error upon running the unit test:

selenium.common.exceptions.WebDriverException: 
Message: unknown error: Chrome failed to start: exited abnormally. 
(chrome not reachable)

I used the same Chrome options as when I'm running my AWS Lambda (which works):

lambda_options = [
    '--autoplay-policy=user-gesture-required',
    '--disable-background-networking',
    '--disable-background-timer-throttling',
    '--disable-backgrounding-occluded-windows',
    '--disable-breakpad',
    '--disable-client-side-phishing-detection',
    '--disable-component-update',
    '--disable-default-apps',
    '--disable-dev-shm-usage',
    '--disable-domain-reliability',
    '--disable-extensions',
    '--disable-features=AudioServiceOutOfProcess',
    '--disable-hang-monitor',
    '--disable-ipc-flooding-protection',
    '--disable-notifications',
    '--disable-offer-store-unmasked-wallet-cards',
    '--disable-popup-blocking',
    '--disable-print-preview',
    '--disable-prompt-on-repost',
    '--disable-renderer-backgrounding',
    '--disable-setuid-sandbox',
    '--disable-speech-api',
    '--disable-sync',
    '--disk-cache-size=33554432',
    '--hide-scrollbars',
    '--ignore-gpu-blacklist',
    '--ignore-certificate-errors',
    '--metrics-recording-only',
    '--mute-audio',
    '--no-default-browser-check',
    '--no-first-run',
    '--no-pings',
    '--no-sandbox',
    '--no-zygote',
    '--password-store=basic',
    '--use-gl=swiftshader',
    '--use-mock-keychain',
    '--single-process',
    '--headless']

for argument in lambda_options:
    chrome_options.add_argument(argument)

With some additional options that are Lambda specific:

chrome_options.add_argument('--user-data-dir={}'.format(tmp_folder + '/user-data'))
chrome_options.add_argument('--data-path={}'.format(tmp_folder + '/data-path'))
chrome_options.add_argument('--homedir={}'.format(tmp_folder))
chrome_options.add_argument('--disk-cache-dir={}'.format(tmp_folder + '/cache-dir'))

chrome_options.binary_location = "/opt/bin/chromium"

tmp_folder is the file system provided by AWS where tmp_folder = /tmp

Lastly, I'm starting the driver:

driver = webdriver.Chrome(options=chrome_options)

This works perfectly well in the Lambda but fails every time I try to run the unit test in GitHub actions.

This is the configuration when running the job in GitHub actions:

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v2
      - name: Set up Python 3.7
        uses: actions/setup-python@v2
        with:
          python-version: 3.7
      - name: Install dependencies
        run: |
        .......

      - name: Unit test
        run: |

          wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add -
          echo "deb http://dl.google.com/linux/chrome/deb/ stable main" | sudo tee -a /etc/apt/sources.list.d/google-chrome.list
          sudo apt-get update -qqy
          sudo apt-get -qqy install google-chrome-stable
          CHROME_VERSION=$(google-chrome-stable --version)
          CHROME_FULL_VERSION=${CHROME_VERSION%%.*}
          CHROME_MAJOR_VERSION=${CHROME_FULL_VERSION//[!0-9]}
          sudo rm /etc/apt/sources.list.d/google-chrome.list
          export CHROMEDRIVER_VERSION=`curl -s https://chromedriver.storage.googleapis.com/LATEST_RELEASE_${CHROME_MAJOR_VERSION%%.*}`
          curl -L -O "https://chromedriver.storage.googleapis.com/${CHROMEDRIVER_VERSION}/chromedriver_linux64.zip"
          unzip chromedriver_linux64.zip && chmod +x chromedriver && sudo mv chromedriver /usr/local/bin
          export CHROMEDRIVER_VERSION=`curl -s https://chromedriver.storage.googleapis.com/LATEST_RELEASE_${CHROME_MAJOR_VERSION%%.*}`
          curl -L -O "https://chromedriver.storage.googleapis.com/${CHROMEDRIVER_VERSION}/chromedriver_linux64.zip"
          unzip chromedriver_linux64.zip && chmod +x chromedriver && sudo mv chromedriver /usr/local/bin
          chromedriver -version
          which chromedriver
          which google-chrome

The configuration entirely is taken from the Chrome setup in GibHub actions made by Selenium themselves (https://github.com/SeleniumHQ/selenium/blob/selenium-4.0.0-beta-3/.github/actions/setup-chrome/action.yml), and it works great.

This GitHub actions configuration is using the latest ChromeDriver, so I've both tried older versions (95.0.4638.54 as mentioned before) and the latest one (96.0.4664.45), but I'm still getting the same error:

selenium.common.exceptions.WebDriverException: 
Message: unknown error: Chrome failed to start: exited abnormally. 
(chrome not reachable)

Answer 1

Here's what I needed to add to the Chrome options to make it work.

First add a remote debugging port:

chrome_options.add_argument('--remote-debugging-port=9222')

Then I also changed the binary location to the following one, the binary_location option is for Google Chrome:

chrome_options.binary_location = "/usr/bin/google-chrome"

And finally, as mentioned in this answer (https://stackoverflow.com/a/60092331/6697714), I also added an executable path to the Chrome driver:

executable_path="/usr/local/bin/chromedriver"

The lambda config is kept as before:

lambda_options = [
    '--autoplay-policy=user-gesture-required',
    '--disable-background-networking',
    '--disable-background-timer-throttling',
    '--disable-backgrounding-occluded-windows',
    '--disable-breakpad',
    '--disable-client-side-phishing-detection',
    '--disable-component-update',
    '--disable-default-apps',
    '--disable-dev-shm-usage',
    '--disable-domain-reliability',
    '--disable-extensions',
    '--disable-features=AudioServiceOutOfProcess',
    '--disable-hang-monitor',
    '--disable-ipc-flooding-protection',
    '--disable-notifications',
    '--disable-offer-store-unmasked-wallet-cards',
    '--disable-popup-blocking',
    '--disable-print-preview',
    '--disable-prompt-on-repost',
    '--disable-renderer-backgrounding',
    '--disable-setuid-sandbox',
    '--disable-speech-api',
    '--disable-sync',
    '--disk-cache-size=33554432',
    '--hide-scrollbars',
    '--ignore-gpu-blacklist',
    '--ignore-certificate-errors',
    '--metrics-recording-only',
    '--mute-audio',
    '--no-default-browser-check',
    '--no-first-run',
    '--no-pings',
    '--no-sandbox',
    '--no-zygote',
    '--password-store=basic',
    '--use-gl=swiftshader',
    '--use-mock-keychain',
    '--single-process',
    '--headless',
]

for argument in lambda_options:
    chrome_options.add_argument(argument)

Here's how the final config looks like:

chrome_options.add_argument('--disable-gpu')
chrome_options.add_argument('--user-data-dir={}'.format(tmp_folder + '/user-data'))
chrome_options.add_argument('--data-path={}'.format(tmp_folder + '/data-path'))
chrome_options.add_argument('--homedir={}'.format(tmp_folder))
chrome_options.add_argument('--disk-cache-dir={}'.format(tmp_folder + '/cache-dir'))
chrome_options.add_argument('--remote-debugging-port=9222')

chrome_options.binary_location = "/usr/bin/google-chrome"
driver = webdriver.Chrome(options=chrome_options, executable_path="/usr/local/bin/chromedriver")