I have a Python script that is called from PHP using shell_exec, all was working fine and the script came back to the PHP function after the Python script completed and everything was working as exepcted.
Now I have changed the Python script to include a class one level up from it that creates a web driver and returns the driver to the script that calls it via an object and then another function closes the driver that is passed in as a parameter.
Since setting up this new class and adding in an object to create/close a driver for webdriver using chrome the Python script hangs after completion and it never hands back control to the PHP script to finish processing.
I've had a look at php system() shell_exec() hangs the browser and also at http://php.net/manual/it/function.shell-exec.php#106250 with no luck.
I need the PHP script to run and then pass to the Python script, wait for the Python script to complete and hand back to the PHP script to finish processing.
All this process is kicked off from Run.php with:
$reponse = shell_exec('python36 "script1.py" "https://www.example.com/" "23"');
Directory Structure
root_dir
python_dir
scripts_dir
Script1.py
Script2.py
Script3.py
Global.py
run.php
Global.py
import re
from selenium import webdriver
class Global:
def getDriver(self):
try:
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('headless')
chrome_options.add_argument('no-sandbox')
# Windows - Dev Environment
driver = webdriver.Chrome(executable_path='C:\chromedriver.exe', chrome_options=chrome_options)
# Linux - Prod Environment
#driver = webdriver.Chrome(chrome_options=chrome_options)
return driver
except Exception as e:
print('Error: ' + str(e.args[0]))
def closeDriver(self, driver):
try:
driver.close()
return None
except Exception as e:
print('Error: ' + str(e.args[0]))
All the scripts in the scripts_dir have different code in them depeneding on the task they are doing, but the new Global object stuff is the same for all scripts, the new Global object is the only change made to all the scripts since they were working fine till now where the Python scripts hang.
Script1.py
import os, sys
ROOT_DIR = os.path.normpath(os.path.join(os.path.abspath(__file__), '../..'))
sys.path.insert(0, ROOT_DIR)
from Global import Global
globalObj = Global()
try:
# Load all items on page
driver = globalObj.getDriver()
driver.get(PARAMS[1])
#... code to create a JSON file and populate it.
# Close webdriver
globalObj.closeDriver(driver)
print('Completed Ref: ' + str(PARAMS[2]))
except Exception as e:
print('Error: ' + str(e.args[0]))
There are no errors or warnings in Python or PHP or any other logs available to me, the Python scripts run and create the expected JSON files they should and all looks to be working normally, the line print('Completed Ref: ' + str(PARAMS[2])) doesn't seem to print out when the Python script hangs.
Running Script1.py on the command line takes about 35 seconds to complete successfully.
If I remove the calls to the Global class to getDriver and closeDriver then the script runs and completes and hands back to PHP to continue processing.
Script1.py - Update
import os, sys
from selenium import webdriver
try:
# Load all items on page
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('headless')
chrome_options.add_argument('no-sandbox')
driver = webdriver.Chrome(executable_path='C:\chromedriver.exe', chrome_options=chrome_options)
driver.get(PARAMS[1])
#... code to create a JSON file and populate it.
# Close webdriver
driver.close()
print('Completed Ref: ' + str(PARAMS[2]))
except Exception as e:
print('Error: ' + str(e.args[0]))
The script runs fine in both command line and when executed via PHP using shell_exec using the above example, so there must be something I'm missing when it comes to creating an object and call the getDriver or closeDriver classes.
After further testing, I can confirm that the issue is being caused by the script1.py script calling the class Global to create the webdriver and close it, once I remove these calls in script1.py the script runs fine.
I've tried putting in sys.exit() after the globalObj.closeDriver(), I've also tried del globalObj at the end of the script with no luck.
I've changed out the webdriver from Chrome to Firefox and PhantomJS, Other drivers are working fine with the Global class, looking like it might be related to the Chrome driver at this stage.
I'm using Chrome 64.0.3282.186 (Official Build) (64-bit) and Chromedriver 2.35.
It looks like driver.close() is the cause of the problem. You need to use .quit() with Chromedriver to end the chromedriver process otherwise the process will not terminate which is why the Python script looked like it was hanging.
From:
driver.close()
To:
driver.quit()