Azure Machine Learning installing chromedriver on a pipeline

Question

I m trying to run my python script (which uses chromedriver) on a pipeline to be able to call it from Azure Data Factory.

when i run the pipeline, i am getting error of chromedriver isn't at path. The script works fine with my local environment. I also tried to install chromedriver according to below code in my python script.

import os
os.system('sudo apt-get install unzip')
os.system('wget -N https://chromedriver.storage.googleapis.com/94.0.4606.61/chromedriver_linux64.zip -P ~/Downloads')
os.system('unzip ~/Downloads/chromedriver_linux64.zip -d ~/Downloads')
os.system('sudo mv -f ~/Downloads/chromedriver /usr/local/share/')
os.system('sudo chmod +x /usr/local/share/chromedriver')
os.system('sudo ln -s /usr/local/share/chromedriver /usr/local/bin/chromedriver')
os.system('sudo ln -s /usr/local/share/chromedriver /usr/bin/chromedriver')

How i call chrome driver in script:

options= Options()
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')
options.add_argument('window-size=1920,1080')
options.add_argument('--headless')
options.add_experimental_option('w3c', False)
driver = webdriver.Chrome(executable_path=r'/usr/bin/chromedriver',chrome_options=options)

The error when i run pipeline:

ActivityFailedException: ActivityFailedException:
    Message: Activity Failed:
{
    "error": {
        "code": "UserError",
        "message": "User program failed with WebDriverException: Message: 'chromedriver' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home\n",
        "messageParameters": {},
        "detailsUri": "https://aka.ms/azureml-run-troubleshooting",
        "details": []
    },
    "time": "0001-01-01T00:00:00.000Z"
}
    InnerException None
    ErrorResponse 
{
    "error": {
        "message": "Activity Failed:\n{\n    \"error\": {\n        \"code\": \"UserError\",\n        \"message\": \"User program failed with WebDriverException: Message: 'chromedriver' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home\\n\",\n        \"messageParameters\": {},\n        \"detailsUri\": \"https://aka.ms/azureml-run-troubleshooting\",\n        \"details\": []\n    },\n    \"time\": \"0001-01-01T00:00:00.000Z\"\n}"
    }
}

Thank you for your time.

Answer 1

Using dockerfile for creating a new environment and installing chrome with requirements solved the problem.