Search code examples
pythonpippython-wheel

Where are stored wheels .whl cached files?

$ python3 -m venv ~/venvs/vtest
$ source ~/venvs/vtest/bin/activate
(vtest) $ pip install numpy
Collecting numpy
  Cache entry deserialization failed, entry ignored
  Using cached https://files.pythonhosted.org/packages/d2/ab/43e678759326f728de861edbef34b8e2ad1b1490505f20e0d1f0716c3bf4/numpy-1.17.4-cp36-cp36m-manylinux1_x86_64.whl
Installing collected packages: numpy
Successfully installed numpy-1.17.4
(vtest) $

I'm looking for where this wheel numpy-1.17.4-cp36-cp36m-manylinux1_x86_64.whl has been cached ?

$ sudo updatedb
$ locate numpy-1.17.4
$ # nada ;(

Documentation https://pip.pypa.io/en/stable/reference/pip_install/#wheel-cache tell us that Pip will read from the subdirectory wheels within the pip cache directory and use any packages found there.

$ pip --version
pip 9.0.1 from ~/venvs/vtest/lib/python3.6/site-packages (python 3.6)
$

To answer Hamza Khurshid numpy is not on ~/.cache/pip/wheels

$ find ~/.cache/pip/wheels -name '*.whl' |grep -i numpy
$

it look like .cache/pip/wheels is only used for user created wheels not for downloaded wheels, should I use export PIP_DOWNLOAD_CACHE=$HOME/.pip/cache ?

Solution

  • The message

    Using cached https://files.pythonhosted.org/packages/d2/ab/43e678759326f728de861edbef34b8e2ad1b1490505f20e0d1f0716c3bf4/numpy-1.17.4-cp36-cp36m-manylinux1_x86_64.whl

    means pip is using the HTTP cache, not the wheel cache (which is only used for locally-built wheels, like you mentioned).

    The name of the file in the HTTP cache is the sha224 of the URL being requested.

    You can retrieve the file like

    $ pwd
/home/user/.cache/pip/http
$ find . -name "$(printf 'https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c19460
6c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl' | sha224sum - | awk '{print $1}')"
./f/6/0/2/d/f602daffc1b0025a464d60b3e9f8b1f77a4538b550a46d67018978db

    The format of the file is not stable though, and depends on pip version. For specifics you can see the implementation that's used in the latest cachecontrol, which pip uses.

    If you want to get the actual file, an easier way is to use pip download, which will take the file from the cache into your current directory if it matches the URL that would be otherwise downloaded.