Search code examples
pythonpippython-requestsurllib3chalice

pip install urllib3 hanging on "Caching due to etag"

Pip install of urllib3 is hanging on "Caching due to etag". I'm building an AWS chalice project that doesn't let you specify --no-cache-dir, so I need to fix the issue without that command. Any ideas?

Using Python 3.6.5 and Pip 10.0.1 in a virtual environment.

(partnerdb-virtualenv) C:\Windows\SysWOW64\partnerdb-project>pip install urllib3 -vvv
Config variable 'Py_DEBUG' is unset, Python ABI tag may be incorrect
Config variable 'WITH_PYMALLOC' is unset, Python ABI tag may be incorrect
Created temporary directory: C:\Users\Matt\AppData\Local\Temp\pip-ephem-wheel-cache-v0e8ikpl
Created temporary directory: C:\Users\Matt\AppData\Local\Temp\pip-req-tracker-d58lw_h5
Created requirements tracker 'C:\\Users\\Matt\\AppData\\Local\\Temp\\pip-req-tracker-d58lw_h5'
Created temporary directory: C:\Users\Matt\AppData\Local\Temp\pip-install-6qpv92ms
Collecting urllib3
  1 location(s) to search for versions of urllib3:
  * https://pypi.org/simple/urllib3/
  Getting page https://pypi.org/simple/urllib3/
  Looking up "https://pypi.org/simple/urllib3/" in the cache
  Request header has "max_age" as 0, cache bypassed
  Starting new HTTPS connection (1): pypi.org:443
  https://pypi.org:443 "GET /simple/urllib3/ HTTP/1.1" 200 6330
  Updating cache with response from "https://pypi.org/simple/urllib3/"
  Caching due to etag

Edit: Based on one of the comments, curl -vvv https://files/pythonhosted.org/ returns:

*   Trying 2a04:4e42:2d::319...
* TCP_NODELAY set
* Connected to files.pythonhosted.org (2a04:4e42:2d::319) port 443 (#0)
* schannel: SSL/TLS connection with files.pythonhosted.org port 443 (step 1/3)
* schannel: checking server certificate revocation
* schannel: sending initial handshake data: sending 187 bytes...
* schannel: sent initial handshake data: sent 187 bytes
* schannel: SSL/TLS connection with files.pythonhosted.org port 443 (step 2/3)
* schannel: failed to receive handshake, need more data
* schannel: SSL/TLS connection with files.pythonhosted.org port 443 (step 2/3)
* schannel: encrypted data got 4096
* schannel: encrypted data buffer: offset 4096 length 4096
* schannel: encrypted data length: 4018
* schannel: encrypted data buffer: offset 4018 length 4096
* schannel: received incomplete message, need more data
* schannel: SSL/TLS connection with files.pythonhosted.org port 443 (step 2/3)
* schannel: encrypted data got 1024
* schannel: encrypted data buffer: offset 5042 length 5042
* schannel: encrypted data length: 815
* schannel: encrypted data buffer: offset 815 length 5042
* schannel: received incomplete message, need more data
* schannel: SSL/TLS connection with files.pythonhosted.org port 443 (step 2/3)
* schannel: encrypted data got 1051
* schannel: encrypted data buffer: offset 1866 length 5042
* schannel: sending next handshake data: sending 93 bytes...
* schannel: SSL/TLS connection with files.pythonhosted.org port 443 (step 2/3)
* schannel: encrypted data got 258
* schannel: encrypted data buffer: offset 258 length 5042
* schannel: SSL/TLS handshake complete
* schannel: SSL/TLS connection with files.pythonhosted.org port 443 (step 3/3)
* schannel: stored credential handle in session cache
> GET / HTTP/1.1
> Host: files.pythonhosted.org
> User-Agent: curl/7.55.1
> Accept: */*
>
* schannel: client wants to read 102400 bytes
* schannel: encdata_buffer resized 103424
* schannel: encrypted data buffer: offset 0 length 103424
* schannel: encrypted data got 2412
* schannel: encrypted data buffer: offset 2412 length 103424
* schannel: decrypted data length: 2383
* schannel: decrypted data added: 2383
* schannel: decrypted data cached: offset 2383 length 102400
* schannel: encrypted data buffer: offset 0 length 103424
* schannel: decrypted data buffer: offset 2383 length 102400
* schannel: schannel_recv cleanup
* schannel: decrypted data returned 2383
* schannel: decrypted data buffer: offset 0 length 102400
< HTTP/1.1 200 OK
< Content-Type: text/html
< Server: nginx/1.13.9
< Content-Length: 1822
< Accept-Ranges: bytes
< Date: Thu, 18 Oct 2018 18:18:01 GMT
< Age: 0
< Connection: keep-alive
< X-Served-By: cache-iad2138-IAD, cache-pao17439-PAO
< X-Cache: HIT, MISS
< X-Cache-Hits: 1, 0
< X-Timer: S1539886682.749014,VS0,VE70
< Vary: Accept-Encoding
< Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
< X-Frame-Options: deny
< X-XSS-Protection: 1; mode=block
< X-Content-Type-Options: nosniff
< X-Permitted-Cross-Domain-Policies: none
< X-Robots-Header: noindex
<
<html> ... </html>
* Connection #0 to host files.pythonhosted.org left intact
Solution

  • A long and late shot but worth trying:

    I had a similar issue with cache for one package only. Using --no-cache-dir everything worked fine but this was not a solution.

    I found this answer, and deleted the whole pip cache, bang!

    From the docs:

    The default location for the cache directory depends on the Operating System:

    Unix

    ~/.cache/pip and it respects the XDG_CACHE_HOME directory.

    macOS

    ~/Library/Caches/pip.

    Windows

    <CSIDL_LOCAL_APPDATA>\pip\Cache