Having quite a difficult time installing Python requirements.txt to a PHP application on Elastic Beanstalk.
Initially I had questioned the ability to Deploy multiple platforms to Elastic Beanstalk (PHP/Python). While this is not possible out of the box, it is possible to run pre-install commands via .ebextentions
This lead to creating an .ebextentions/install_python_requirements.config
container_commands:
python_req:
command: 'pip install -r /var/app/ondeck/requirements.txt'
Trouble is, now lxml, a dependency in requirements, consistently fails during the deploy process. Curiously, ssh directly into the EC2 instance then running pip install -r requirements.txt completes without issue.
Why is the dependency installation succeeding via direct ssh access, but failing during deployment using eb deploy and the install_python_requirements.config?
Failure in /var/log/eb-activity.log
creating build/temp.linux-x86_64-2.7/src/lxml
gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/libxml2 -I/tmp/pip-build-A6NhcA/lxml/src/lxml/includes -I/usr/include/python2.7 -c src/lxml/lxml.etree.c -o build/temp.linux-x86_64-2.7/src/lxml/lxml.etree.o -w
gcc: error trying to exec 'cc1': execvp: No such file or directory
error: command 'gcc' failed with exit status 1
----------------------------------------
Command "/usr/bin/python2.7 -c "import setuptools, tokenize;__file__='/tmp/pip-build-A6NhcA/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-gSsRbZ-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-A6NhcA/lxml
(ElasticBeanstalk::ExternalInvocationError)
As it turns out, the issue ended up being the system $PATH. It appears that during eb deploy the system path is unset. The pip install is able to begin, as the system PATH is available to the deploy process. However, that system PATH variable is not passed into the pip process. Therefore, when subsequent calls to pip are attempted, they fail because they cannot find the application path. As evidenced by adding a log to the install_python_requirements.config
whome:
command: 'whoami'
env:
command: '/bin/sh -c env'
Then checking /var/log/eb-activity.log for the output of the log commands above, and noticing the absense of a $PATH
The solution for this was setting the PATH manually via another .ebextentions/var.config
option_settings:
- option_name: PATH
value: '/usr/local/sbin:/sbin:/bin:/usr/sbin:/usr/bin:/opt/aws/bin:/root/bin'
This allowed the requirements.txt to complete successfully during the eb deploy process