I am trying to run a simple scraper on Heroku. Getting this error.
2021-06-24T13:18:50.881101+00:00 heroku[web.1]: Starting process with command `python3 spotlight.py`
2021-06-24T13:18:55.146843+00:00 app[web.1]: /app/.heroku/python/lib/python3.9/site-packages/selenium/webdriver/firefox/firefox_profile.py:208: SyntaxWarning: "is" with a literal. Did you mean "=="?
2021-06-24T13:18:55.146853+00:00 app[web.1]: if setting is None or setting is '':
2021-06-24T13:18:55.320042+00:00 app[web.1]: Traceback (most recent call last):
2021-06-24T13:18:55.320112+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.9/site-packages/selenium/webdriver/common/service.py", line 72, in start
2021-06-24T13:18:55.320559+00:00 app[web.1]: self.process = subprocess.Popen(cmd, env=self.env,
2021-06-24T13:18:55.320586+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.9/subprocess.py", line 951, in __init__
2021-06-24T13:18:55.321268+00:00 app[web.1]: self._execute_child(args, executable, preexec_fn, close_fds,
2021-06-24T13:18:55.321291+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.9/subprocess.py", line 1821, in _execute_child
2021-06-24T13:18:55.323124+00:00 app[web.1]: raise child_exception_type(errno_num, err_msg, err_filename)
2021-06-24T13:18:55.323253+00:00 app[web.1]: FileNotFoundError: [Errno 2] No such file or directory: 'None'
2021-06-24T13:18:55.323272+00:00 app[web.1]:
2021-06-24T13:18:55.323273+00:00 app[web.1]: During handling of the above exception, another exception occurred:
2021-06-24T13:18:55.323273+00:00 app[web.1]:
2021-06-24T13:18:55.323293+00:00 app[web.1]: Traceback (most recent call last):
2021-06-24T13:18:55.323352+00:00 app[web.1]: File "/app/spotlight.py", line 17, in <module>
2021-06-24T13:18:55.323614+00:00 app[web.1]: driver = webdriver.Chrome(executable_path=str(os.environ.get('CHROMEDRIVER_PATH')),options=options1)
2021-06-24T13:18:55.323641+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.9/site-packages/selenium/webdriver/chrome/webdriver.py", line 73, in __init__
2021-06-24T13:18:55.323953+00:00 app[web.1]: self.service.start()
2021-06-24T13:18:55.323975+00:00 app[web.1]: File "/app/.heroku/python/lib/python3.9/site-packages/selenium/webdriver/common/service.py", line 81, in start
2021-06-24T13:18:55.324281+00:00 app[web.1]: raise WebDriverException(
2021-06-24T13:18:55.324441+00:00 app[web.1]: selenium.common.exceptions.WebDriverException: Message: 'None' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home
2021-06-24T13:18:55.324475+00:00 app[web.1]:
2021-06-24T13:18:55.472400+00:00 heroku[web.1]: Process exited with status 1
2021-06-24T13:18:55.569966+00:00 heroku[web.1]: State changed from starting to crashed
I have set up the build packs and using this code for the paths
import os
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
GOOGLE_CHROME_PATH = '/app/.apt/usr/bin/google_chrome'
CHROMEDRIVER_PATH = '/app/.chromedriver/bin/chromedriver'
options1 = Options()
options1.binary_location = os.environ.get('GOOGLE_CHROME_BIN')
options1.add_argument("--headless")
options1.add_argument("--example-flag")
options1.add_argument('--no-sandbox')
options1.add_argument('--disable-dev-shm-usage')
driver = webdriver.Chrome(executable_path=str(os.environ.get('CHROMEDRIVER_PATH')),options=options1)
driver.maximize_window()
Why is there a firefox warning?
The same code was working fine for other app. Cannot understand this error or the fix. Can someone help?
Pipfile
[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"
[packages]
selenium = "*"
requests = "*"
Pillow = "*"
[dev-packages]
[requires]
python_version = "3.9"
For anyone else facing the same issue, You need to set the environment variables from the cli like so,
heroku config:set GOOGLE_CHROME_BIN=/app/.apt/usr/bin/google-chrome
heroku config:set CHROMEDRIVER_PATH=/app/.chromedriver/bin/chromedriver