I've been trying to install pycurl in a virtualenv with easy_install, and it appears to install correctly:
(xxx) $ easy_install pycurl
Searching for pycurl
Reading http://pypi.python.org/simple/pycurl/
Reading http://pycurl.sourceforge.net/
Reading http://pycurl.sourceforge.net/download/
Best match: pycurl 7.19.0
Downloading http://pycurl.sourceforge.net/download/pycurl-7.19.0.tar.gz
Processing pycurl-7.19.0.tar.gz
Running pycurl-7.19.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-B2W9dY/pycurl-7.19.0/egg-dist-tmp-RmIsVr
Using curl-config (libcurl 7.19.7)
src/pycurl.c:85:4: warning: #warning "libcurl was compiled with SSL support, but configure could not determine which " "library was used; thus no SSL crypto locking callbacks will be set, which may " "cause random crashes on SSL requests"
src/pycurl.c: In function ‘do_multi_info_read’:
src/pycurl.c:2843: warning: call to ‘_curl_easy_getinfo_err_string’ declared with attribute warning: curl_easy_getinfo expects a pointer to char * for this info
src/pycurl.c: In function ‘multi_socket_callback’:
src/pycurl.c:2355: warning: call to ‘_curl_easy_getinfo_err_string’ declared with attribute warning: curl_easy_getinfo expects a pointer to char * for this info
In function ‘util_curl_unsetopt’,
inlined from ‘do_curl_unsetopt’ at src/pycurl.c:1551:
src/pycurl.c:1476: warning: call to ‘_curl_easy_setopt_err_CURLSH’ declared with attribute warning: curl_easy_setopt expects a CURLSH* argument for this option
gcc (GCC) 4.4.4 20100726 (Red Hat 4.4.4-13)
Copyright (C) 2010 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
zip_safe flag not set; analyzing archive contents...
Adding pycurl 7.19.0 to easy-install.pth file
Installed /home/ec2-user/.virtualenvs/xxx/lib/python2.7/site-packages/pycurl-7.19.0-py2.7-linux-x86_64.egg
Processing dependencies for pycurl
Finished processing dependencies for pycurl
However, when trying to use, I get the following error:
(xxx) $ python
Python 2.7.1 (r271:86832, Sep 12 2011, 13:51:42)
[GCC 4.4.4 20100726 (Red Hat 4.4.4-13)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pycurl
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named pycurl
Despite the fact that I can clearly see a pycurl-7.19.0-py2.7-linux-x86_64.egg in my ~/.virtualenvs/xxx/lib/python2.7/site-packages directory.
Any suggestions on how to solve this problem would be much appreciated!
Distribution is Amazon Linux (EC2, 64 bit), Python is 2.7, installed from source. curl, libcurl and libcurl-dev are all supplied by the Amazon yum repos and are at version 7.19.7-26.20.
I can't install pycurl package provided by the Amazon repos because the distro-supplied version of python is 2.6, and my project runs on 2.7.
It is not a path issue, the contents of sys.path is as follows:
>>> import sys; sys.path
['', '/home/ec2-user/.virtualenvs/xxx/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg', '/home/ec2-user/.virtualenvs/xxx/lib/python2.7/site-packages/pip-1.0.2-py2.7.egg', '/home/ec2-user/.virtualenvs/xxx/lib/python2.7/site-packages/pygeoip-0.2.1-py2.7.egg', '/home/ec2-user/.virtualenvs/xxx/lib/python2.7/site-packages/daryl_lib-1.8.4-py2.7.egg', '/home/ec2-user/.virtualenvs/xxx/lib/python2.7/site-packages/xxx-1.8.4-py2.7.egg', '/home/ec2-user/dealutils-1.5.0-py2.7.egg', '/home/ec2-user/.virtualenvs/xxx/lib/python2.7/site-packages/six-1.0.0-py2.7.egg', '/home/ec2-user/fm2c-1.3.0-py2.7.egg', '/home/ec2-user/.virtualenvs/xxx/lib/python2.7/site-packages/affiliatewindow-1.3.0-py2.7.egg', '/home/ec2-user/.virtualenvs/xxx/lib/python2.7/site-packages/icodes-1.3.0-py2.7.egg', '/home/ec2-user/.virtualenvs/xxx/lib/python2.7/site-packages/supervisor-3.0a10-py2.7.egg', '/home/ec2-user/.virtualenvs/xxx/lib/python2.7/site-packages/meld3-0.6.7-py2.7.egg', '/home/ec2-user/.virtualenvs/xxx/lib/python2.7/site-packages/tornado-1.2-py2.7.egg', '/home/ec2-user/.virtualenvs/xxx/lib/python2.7/site-packages/pycurl-7.19.0-py2.7-linux-x86_64.egg', '/home/ec2-user/.virtualenvs/xxx/lib/python2.7/site-packages', '/root', '/home/ec2-user/.virtualenvs/xxx/lib/python27.zip', '/home/ec2-user/.virtualenvs/xxx/lib/python2.7', '/home/ec2-user/.virtualenvs/xxx/lib/python2.7/plat-linux2', '/home/ec2-user/.virtualenvs/xxx/lib/python2.7/lib-tk', '/home/ec2-user/.virtualenvs/xxx/lib/python2.7/lib-old', '/home/ec2-user/.virtualenvs/xxx/lib/python2.7/lib-dynload', '/opt/python2.7/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg', '/opt/python2.7/lib/python2.7/site-packages/pip-1.0.2-py2.7.egg', '/opt/python2.7/lib/python2.7', '/opt/python2.7/lib/python2.7/plat-linux2', '/opt/python2.7/lib/python2.7/lib-tk', '/opt/python2.7/lib/python2.7/site-packages']
I have managed (with some help) so solve the problem. The problem is that after installation, the pycurl.so is not copied to the site-packages for this virtualenv.
When installing via pip -v install pycurl, the following output is given:
Downloading/unpacking pycurl
Using version 7.19.0 (newest of versions: 7.19.0, 7.19.0, 7.18.2, 7.18.1, 7.16.4, 7.16.2.1, 7.16.2, 7.16.1, 7.15.5.1)
Downloading pycurl-7.19.0.tar.gz (71Kb): 71Kb downloaded
Running setup.py egg_info for package pycurl
Using curl-config (libcurl 7.19.7)
running egg_info
creating pip-egg-info/pycurl.egg-info
writing pip-egg-info/pycurl.egg-info/PKG-INFO
writing top-level names to pip-egg-info/pycurl.egg-info/top_level.txt
writing dependency_links to pip-egg-info/pycurl.egg-info/dependency_links.txt
writing manifest file 'pip-egg-info/pycurl.egg-info/SOURCES.txt'
warning: manifest_maker: standard file '-c' not found
reading manifest file 'pip-egg-info/pycurl.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'pip-egg-info/pycurl.egg-info/SOURCES.txt'
Installing collected packages: pycurl
Running setup.py install for pycurl
Using curl-config (libcurl 7.19.7)
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-2.7
creating build/lib.linux-x86_64-2.7/curl
copying python/curl/__init__.py -> build/lib.linux-x86_64-2.7/curl
running build_ext
building 'pycurl' extension
creating build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/src
gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DHAVE_CURL_SSL=1 -I/opt/python2.7/include/python2.7 -c src/pycurl.c -o build/temp.linux-x86_64-2.7/src/pycurl.o
src/pycurl.c:85:4: warning: #warning "libcurl was compiled with SSL support, but configure could not determine which " "library was used; thus no SSL crypto locking callbacks will be set, which may " "cause random crashes on SSL requests"
src/pycurl.c: In function ‘do_multi_info_read’:
src/pycurl.c:2843: warning: call to ‘_curl_easy_getinfo_err_string’ declared with attribute warning: curl_easy_getinfo expects a pointer to char * for this info
src/pycurl.c: In function ‘multi_socket_callback’:
src/pycurl.c:2355: warning: call to ‘_curl_easy_getinfo_err_string’ declared with attribute warning: curl_easy_getinfo expects a pointer to char * for this info
In function ‘util_curl_unsetopt’,
inlined from ‘do_curl_unsetopt’ at src/pycurl.c:1551:
src/pycurl.c:1476: warning: call to ‘_curl_easy_setopt_err_CURLSH’ declared with attribute warning: curl_easy_setopt expects a CURLSH* argument for this option
gcc -pthread -shared build/temp.linux-x86_64-2.7/src/pycurl.o -L/opt/python2.7/lib -lcurl -lpython2.7 -o build/lib.linux-x86_64-2.7/pycurl.so unknown option: --static-libs Usage: curl-config [OPTION] Available values for OPTION include: --ca ca bundle install path --cc compiler --cflags pre-processor and compiler flags --checkfor [version] check for (lib)curl of the specified version --features newline separated list of enabled features --help display this help and exit --libs library linking information --prefix curl install prefix --protocols newline separated list of enabled protocols --version output version information --vernum output the version information as a number (hexadecimal)
gcc (GCC) 4.4.4 20100726 (Red Hat 4.4.4-13)
Copyright (C) 2010 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
running install_lib
creating /home/ec2-user/.virtualenvs/xxx/lib/python2.7/site-packages/curl
copying build/lib.linux-x86_64-2.7/curl/__init__.py -> /home/ec2-user/.virtualenvs/xxx/lib/python2.7/site-packages/curl
byte-compiling /home/ec2-user/.virtualenvs/xxx/lib/python2.7/site-packages/curl/__init__.py to __init__.pyc
running install_data
copying ChangeLog -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl
copying COPYING -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl
copying COPYING2 -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl
copying INSTALL -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl
copying README -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl
copying TODO -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl
copying doc/curlshareobject.html -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/html
copying doc/pycurl.html -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/html
copying doc/curlobject.html -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/html
copying doc/curlmultiobject.html -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/html
copying doc/callbacks.html -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/html
copying examples/retriever.py -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/examples
copying examples/file_upload.py -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/examples
copying examples/linksys.py -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/examples
copying examples/xmlrpc_curl.py -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/examples
copying examples/retriever-multi.py -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/examples
copying examples/sfquery.py -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/examples
copying examples/basicfirst.py -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/examples
copying tests/test.py -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/tests
copying tests/test_ftp.py -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/tests
copying tests/test_share.py -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/tests
copying tests/test_multi5.py -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/tests
copying tests/test_multi_timer.py -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/tests
copying tests/test_post.py -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/tests
copying tests/test_gtk.py -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/tests
copying tests/test_post3.py -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/tests
copying tests/test_debug.py -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/tests
copying tests/test_multi4.py -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/tests
copying tests/test_cb.py -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/tests
copying tests/test_multi_vs_thread.py -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/tests
copying tests/test_stringio.py -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/tests
copying tests/test_xmlrpc.py -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/tests
copying tests/test_multi_socket_select.py -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/tests
copying tests/test_multi6.py -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/tests
copying tests/test_multi_socket.py -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/tests
copying tests/test_multi.py -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/tests
copying tests/util.py -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/tests
copying tests/test_multi3.py -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/tests
copying tests/test_internals.py -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/tests
copying tests/test_post2.py -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/tests
copying tests/test_memleak.py -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/tests
copying tests/test_multi2.py -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/tests
copying tests/test_socketopen.py -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/tests
copying tests/test_getinfo.py -> /home/ec2-user/.virtualenvs/xxx/share/doc/pycurl/tests
running install_egg_info
running egg_info
creating pycurl.egg-info
writing pycurl.egg-info/PKG-INFO
writing top-level names to pycurl.egg-info/top_level.txt
writing dependency_links to pycurl.egg-info/dependency_links.txt
writing manifest file 'pycurl.egg-info/SOURCES.txt'
warning: manifest_maker: standard file '-c' not found
reading manifest file 'pycurl.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'pycurl.egg-info/SOURCES.txt'
Copying pycurl.egg-info to /home/ec2-user/.virtualenvs/xxx/lib/python2.7/site-packages/pycurl-7.19.0-py2.7.egg-info
running install_scripts
writing list of installed files to '/tmp/pip-lEWzoB-record/install-record.txt'
Successfully installed pycurl
Cleaning up...
Removing temporary dir /home/ec2-user/.virtualenvs/xxx/build...
The key error here is:
gcc -pthread -shared build/temp.linux-x86_64-2.7/src/pycurl.o -L/opt/python2.7/lib -lcurl -lpython2.7 -o build/lib.linux-x86_64-2.7/pycurl.so unknown option: --static-libs Usage: curl-config [OPTION] Available values for OPTION include: --ca ca bundle install path --cc compiler --cflags pre-processor and compiler flags --checkfor [version] check for (lib)curl of the specified version --features newline separated list of enabled features --help display this help and exit --libs library linking information --prefix curl install prefix --protocols newline separated list of enabled protocols --version output version information --vernum output the version information as a number (hexadecimal)
Which shows that curl-config is being called with the argument --static-libs, which it fails to recognise and instead prints it's 'usage' output. Most importantly, it is returning a non-error-code output which makes the compiler assume the pycurl.so compilation was successful.
This is described in more detail here
The solution to this problem was to modify the pycurl's setup.py and change line 101 to read --libs instead of --static-libs. This causes pycurl.so to be built and copied to the site-packages directory successfully.