Why Selenium Python script freeze on EC2?

Question

I'm trying to run a script, but when I ran the process freeze. Here is the code

def get_source_content(url):

  driver_path = f"{settings.BASE_DIR}/geckodriver"
  options = FirefoxOptions()
  options.add_argument("--headless")
  options.add_argument("--disable-gpu")
  options.add_argument("--no-sandbox")
  options.add_argument("--single-process")
  options.add_argument("--ignore-certificate-errors")

  driver = webdriver.Firefox(service=FirefoxService(executable_path=driver_path), options=options)

  try:
    driver.get(url)
    WebDriverWait(driver, 3)
    element = driver.find_element(
        "xpath", "//button[@class='sc-beySPh gNAvzR mde-consent-accept-btn']"
    )
    element.click()
    WebDriverWait(driver, 3)
    source = driver.page_source

  except Exception as ex:
    raise ex

  driver.quit()
  return source

The code only freeze deployed on AWS EC2, locally works well.

Answer 1

SOLVED

The problem was related to the display, so adding this extra options arguments:

options.add_argument("--window-size=800,600")
options.add_argument("--disable-dev-shm-usage")
options.add_argument("--enable-automation")

and use a third party library to simulate fake display the problem was fixed

from pyvirtualdisplay import Display
display = Display(visible=0, size=(800, 600))
display.start()
...
display.stop()