We're using Buildout as a way to build our packages here, and we're consistently getting really frustrating timeouts when trying to download eggs by running bin/buildout
:
We have no distributions for python-dateutil that satisfies 'python-dateutil>=1.5'.
While:
Installing python_section.
Getting distribution for 'python-dateutil>=1.5'.
An internal error occured due to a bug in either zc.buildout or in a
recipe being used:
Traceback (most recent call last):
File "/vagrant/eggs/zc.buildout-2.1.1-py2.7.egg/zc/buildout/buildout.py", line 1921, in main
getattr(buildout, command)(args)
File "/vagrant/eggs/zc.buildout-2.1.1-py2.7.egg/zc/buildout/buildout.py", line 602, in install
installed_files = self[part]._call(recipe.install)
File "/vagrant/eggs/zc.buildout-2.1.1-py2.7.egg/zc/buildout/buildout.py", line 1356, in _call
return f()
File "/vagrant/eggs/zc.recipe.egg-2.0.0-py2.7.egg/zc/recipe/egg/egg.py", line 126, in install
reqs, ws = self.working_set()
File "/vagrant/eggs/zc.recipe.egg-2.0.0-py2.7.egg/zc/recipe/egg/egg.py", line 84, in working_set
allow_hosts=self.allow_hosts)
File "/vagrant/eggs/zc.buildout-2.1.1-py2.7.egg/zc/buildout/easy_install.py", line 782, in install
return installer.install(specs, working_set)
File "/vagrant/eggs/zc.buildout-2.1.1-py2.7.egg/zc/buildout/easy_install.py", line 626, in install
for dist in self._get_dist(requirement, ws):
File "/vagrant/eggs/zc.buildout-2.1.1-py2.7.egg/zc/buildout/easy_install.py", line 448, in _get_dist
dist, avail = self._satisfied(requirement)
File "/vagrant/eggs/zc.buildout-2.1.1-py2.7.egg/zc/buildout/easy_install.py", line 204, in _satisfied
return None, self._obtain(req, source)
File "/vagrant/eggs/zc.buildout-2.1.1-py2.7.egg/zc/buildout/easy_install.py", line 372, in _obtain
if index.obtain(requirement) is None:
File "/usr/lib/python2.7/dist-packages/setuptools/package_index.py", line 340, in obtain
self.prescan(); self.find_packages(requirement)
File "/usr/lib/python2.7/dist-packages/setuptools/package_index.py", line 325, in find_packages
self.scan_url(self.index_url + requirement.unsafe_name+'/')
File "/usr/lib/python2.7/dist-packages/setuptools/package_index.py", line 668, in scan_url
self.process_url(url, True)
File "/usr/lib/python2.7/dist-packages/setuptools/package_index.py", line 223, in process_url
page = self.process_index(url, page)
File "/usr/lib/python2.7/dist-packages/setuptools/package_index.py", line 300, in process_index
self.scan_url(new_url)
File "/usr/lib/python2.7/dist-packages/setuptools/package_index.py", line 668, in scan_url
self.process_url(url, True)
File "/usr/lib/python2.7/dist-packages/setuptools/package_index.py", line 201, in process_url
f = self.open_url(url, "Download error on %s: %%s -- Some packages may not be found!" % url)
File "/usr/lib/python2.7/dist-packages/setuptools/package_index.py", line 610, in open_url
return open_with_auth(url)
File "/usr/lib/python2.7/dist-packages/setuptools/package_index.py", line 753, in _socket_timeout
return func(*args, **kwargs)
File "/usr/lib/python2.7/dist-packages/setuptools/package_index.py", line 779, in open_with_auth
fp = urllib2.urlopen(request)
File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen
return _opener.open(url, data, timeout)
File "/usr/lib/python2.7/urllib2.py", line 400, in open
response = self._open(req, data)
File "/usr/lib/python2.7/urllib2.py", line 418, in _open
'_open', req)
File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain
result = func(*args)
File "/usr/lib/python2.7/urllib2.py", line 1207, in http_open
return self.do_open(httplib.HTTPConnection, req)
File "/usr/lib/python2.7/urllib2.py", line 1180, in do_open
r = h.getresponse(buffering=True)
File "/usr/lib/python2.7/httplib.py", line 1030, in getresponse
response.begin()
File "/usr/lib/python2.7/httplib.py", line 407, in begin
version, status, reason = self._read_status()
File "/usr/lib/python2.7/httplib.py", line 365, in _read_status
line = self.fp.readline()
File "/usr/lib/python2.7/socket.py", line 447, in readline
data = self._sock.recv(self._rbufsize)
timeout: timed out
Is there a way to prevent these timeouts? why do they happen?
The index server failed to respond, the connection timed out. This happens when PyPI is down, for example. By default, the index server is http://pypi.python.org/simple
, but you can set your own with the [buildout]
option index
.
We use an egg proxy (set up with collective.eggproxy
) to create a local cache automatically, to avoid such problems. Our buildouts use that server as the index, and the proxy will download any missing packages from PyPI and cache them locally. So, for any packages that we have used in the past, we can continue to serve even when PyPI is (temporarily) down.