Search code examples
pandaspippython-wheel

PIP3 - How to reduce time taken for Building wheel for <lib>?

I run pip3 on a Jetson Nano board which has 4 cores ARM A57 and class 1 sdcard. It is ubuntu 18.04 with nVidia repositories.

pip3 install --user pandas

Takes almost 1 hour! I have to do it multiple times because I'm building an environment over main OS and dockers. (same behavior on ubuntu and docker image)

HTOP shows that only 1 core is used at 100%; 3 others are idling.

I've thought using --user would resume "Building Wheel" process... Any option to get pip3 things go faster? Can I force Pip3 on multiple cores? It seems to be a GCC task.

Solution

  • Download source code, run python setup.py bdist_wheel to generate a binary wheel once, upload the wheel to some internal server and install from it every time you need it:

    pip install http://<internal-server>/path/to/wheels/pandas.whl

    Binary wheels are installed quite fast; you only need to compile once.

    Or run pip wheel pandas. Or collect all your requirements into requirements.txt and run pip wheel -r requirements.txt to build your wheelhouse. To install from it:

    pip install --index-url=http://<internal-server>/path/to/wheels/ pandas