Search code examples
python-3.xamazon-ec2lxmlec2-ami

lxml is not getting installed on Amazon Linux 64 bit with Python 3.4.3.


NOTE: Duplicate posts didn't solve my problem.

I am using Python 3.4 for a Flask based app on Amazon EC2 instance (64bit Amazon Linux 2016.09 v2.2.0 running Python 3.4).

When deploying lxml is not getting installed. The error log is:

  copying src/lxml/ElementInclude.py -> build/lib.linux-x86_64-3.4/lxml
  copying src/lxml/sax.py -> build/lib.linux-x86_64-3.4/lxml
  copying src/lxml/usedoctest.py -> build/lib.linux-x86_64-3.4/lxml
  copying src/lxml/builder.py -> build/lib.linux-x86_64-3.4/lxml
  copying src/lxml/__init__.py -> build/lib.linux-x86_64-3.4/lxml
  copying src/lxml/pyclasslookup.py -> build/lib.linux-x86_64-3.4/lxml
  creating build/lib.linux-x86_64-3.4/lxml/includes
  copying src/lxml/includes/__init__.py -> build/lib.linux-x86_64-3.4/lxml/includes
  creating build/lib.linux-x86_64-3.4/lxml/html
  copying src/lxml/html/formfill.py -> build/lib.linux-x86_64-3.4/lxml/html
  copying src/lxml/html/soupparser.py -> build/lib.linux-x86_64-3.4/lxml/html
  copying src/lxml/html/_setmixin.py -> build/lib.linux-x86_64-3.4/lxml/html
  copying src/lxml/html/_diffcommand.py -> build/lib.linux-x86_64-3.4/lxml/html
  copying src/lxml/html/ElementSoup.py -> build/lib.linux-x86_64-3.4/lxml/html
  copying src/lxml/html/usedoctest.py -> build/lib.linux-x86_64-3.4/lxml/html
  copying src/lxml/html/builder.py -> build/lib.linux-x86_64-3.4/lxml/html
  copying src/lxml/html/clean.py -> build/lib.linux-x86_64-3.4/lxml/html
  copying src/lxml/html/_html5builder.py -> build/lib.linux-x86_64-3.4/lxml/html
  copying src/lxml/html/diff.py -> build/lib.linux-x86_64-3.4/lxml/html
  copying src/lxml/html/__init__.py -> build/lib.linux-x86_64-3.4/lxml/html
  copying src/lxml/html/defs.py -> build/lib.linux-x86_64-3.4/lxml/html
  copying src/lxml/html/html5parser.py -> build/lib.linux-x86_64-3.4/lxml/html
  creating build/lib.linux-x86_64-3.4/lxml/isoschematron
  copying src/lxml/isoschematron/__init__.py -> build/lib.linux-x86_64-3.4/lxml/isoschematron
  copying src/lxml/lxml.etree.h -> build/lib.linux-x86_64-3.4/lxml
  copying src/lxml/lxml.etree_api.h -> build/lib.linux-x86_64-3.4/lxml
  copying src/lxml/includes/xslt.pxd -> build/lib.linux-x86_64-3.4/lxml/includes
  copying src/lxml/includes/htmlparser.pxd -> build/lib.linux-x86_64-3.4/lxml/includes
  copying src/lxml/includes/dtdvalid.pxd -> build/lib.linux-x86_64-3.4/lxml/includes
  copying src/lxml/includes/config.pxd -> build/lib.linux-x86_64-3.4/lxml/includes
  copying src/lxml/includes/xmlparser.pxd -> build/lib.linux-x86_64-3.4/lxml/includes
  copying src/lxml/includes/xpath.pxd -> build/lib.linux-x86_64-3.4/lxml/includes
  copying src/lxml/includes/uri.pxd -> build/lib.linux-x86_64-3.4/lxml/includes
  copying src/lxml/includes/c14n.pxd -> build/lib.linux-x86_64-3.4/lxml/includes
  copying src/lxml/includes/relaxng.pxd -> build/lib.linux-x86_64-3.4/lxml/includes
  copying src/lxml/includes/xmlschema.pxd -> build/lib.linux-x86_64-3.4/lxml/includes
  copying src/lxml/includes/etreepublic.pxd -> build/lib.linux-x86_64-3.4/lxml/includes
  copying src/lxml/includes/tree.pxd -> build/lib.linux-x86_64-3.4/lxml/includes
  copying src/lxml/includes/xinclude.pxd -> build/lib.linux-x86_64-3.4/lxml/includes
  copying src/lxml/includes/schematron.pxd -> build/lib.linux-x86_64-3.4/lxml/includes
  copying src/lxml/includes/xmlerror.pxd -> build/lib.linux-x86_64-3.4/lxml/includes
  copying src/lxml/includes/etree_defs.h -> build/lib.linux-x86_64-3.4/lxml/includes
  copying src/lxml/includes/lxml-version.h -> build/lib.linux-x86_64-3.4/lxml/includes
  creating build/lib.linux-x86_64-3.4/lxml/isoschematron/resources
  creating build/lib.linux-x86_64-3.4/lxml/isoschematron/resources/rng
  copying src/lxml/isoschematron/resources/rng/iso-schematron.rng -> build/lib.linux-x86_64-3.4/lxml/isoschematron/resources/rng
  creating build/lib.linux-x86_64-3.4/lxml/isoschematron/resources/xsl
  copying src/lxml/isoschematron/resources/xsl/XSD2Schtrn.xsl -> build/lib.linux-x86_64-3.4/lxml/isoschematron/resources/xsl
  copying src/lxml/isoschematron/resources/xsl/RNG2Schtrn.xsl -> build/lib.linux-x86_64-3.4/lxml/isoschematron/resources/xsl
  creating build/lib.linux-x86_64-3.4/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
  copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_abstract_expand.xsl -> build/lib.linux-x86_64-3.4/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
  copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_message.xsl -> build/lib.linux-x86_64-3.4/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
  copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_dsdl_include.xsl -> build/lib.linux-x86_64-3.4/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
  copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_schematron_skeleton_for_xslt1.xsl -> build/lib.linux-x86_64-3.4/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
  copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/iso_svrl_for_xslt1.xsl -> build/lib.linux-x86_64-3.4/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
  copying src/lxml/isoschematron/resources/xsl/iso-schematron-xslt1/readme.txt -> build/lib.linux-x86_64-3.4/lxml/isoschematron/resources/xsl/iso-schematron-xslt1
  running build_ext
  building 'lxml.etree' extension
  creating build/temp.linux-x86_64-3.4
  creating build/temp.linux-x86_64-3.4/src
  creating build/temp.linux-x86_64-3.4/src/lxml
  gcc -pthread -Wno-unused-result -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/libxml2 -Isrc/lxml/includes -I/usr/include/python3.4m -c src/lxml/lxml.etree.c -o build/temp.linux-x86_64-3.4/src/lxml/lxml.etree.o -w
  {standard input}: Assembler messages:
  {standard input}:364136: Error: number of operands mismatch for `movq'
  {standard input}: Error: open CFI at the end of file; missing .cfi_endproc directive
  gcc: internal compiler error: Killed (program cc1)
  Please submit a full bug report,
  with preprocessed source if appropriate.
  See <http://bugzilla.redhat.com/bugzilla> for instructions.
  Compile failed: command 'gcc' failed with exit status 4
  creating tmp
  cc -I/usr/include/libxml2 -I/usr/include/libxml2 -c /tmp/xmlXPathInitxbk9_n9a.c -o tmp/xmlXPathInitxbk9_n9a.o
  cc tmp/xmlXPathInitxbk9_n9a.o -L/usr/lib64 -lxml2 -o a.out
  error: command 'gcc' failed with exit status 4

I have already done yum install libxslt-devel libxml2-devel yum install gcc

Installation of python34-lxml or python3-lxml shows that these packages are not available. I also tried installing python-lxml using yum but show that No package python-lxml available

To install lxml, I connected to EC2 instance using SSH and did sudo pip install lxml but it was also getting terminated.

I finally installed lxml using sudo CFLAGS="-O0" pip install lxml by connecting to EC2 instance using SSH. I have already installed dependencies libxslt-devel libxml2-devel gcc

Still the problem persists and ElasticBeanstalk is showing the above log.

I have seen the following links for same or related problems: Installing lxml module in python lxml not getting installed on AWS Elasticbeanstalk instance Install lxml on Centos 7 - error: command 'gcc' failed with exit status 4 Invalid requirements.txt on deploying to AWS. Pip couldn't install lxml Invalid requirements.txt on deploying to AWS. Pip couldn't install lxml [centos]Fail to pip install lxml, gcc failure Pip install lxml centOSFailed building wheel for lxml


Solution

  • The problem was due to RAM size.

    I increased my RAM and then tried installing it, and it worked.

    If you're facing this problem then install more RAN for your cloud application. lxml takes a good amount of RAM while getting installed.