Search code examples
jenkinspippipenv

use pipenv for running unittest on jenkins takes too long


We migrated to from pip to pipenv.

I think pipenv is great, but ran into a problem.

We run our unittests on every push using git webhook to jenkins. When Using pip, we had a virtual environment and we would activate it and install our requiremnets.txt file before running the tests.

This was pretty fast, pip would run throught the requiremnets that were already met and only install a dependency when one was added.

With Pipenv, I have the Pipfile and the Pipfile.lock in the root of the repo, and running pipenv install seems like its recreating the environment every time, and takes a very long time.

This could be a read deal breaker for us.

Is there any workaroud so I can speed up working with pipenv?

Previous shell script (in the jenkins job) was something like:

eval "$(pyenv init -)"
eval "$(pyenv virtualenv-init -)"
pyenv activate romee

pip install -r requirements.txt
pip install -r requirements-dev.txt

python tests.py

With pipenv:

pipenv install
pipenv install --dev
pipenv run python run_tests.py

Solution

  • It sounds like your older setup retained the virtual environment and prior pip installed packages across builds and your new pipenv setup is not.

    By default, pipenv creates a virtual environment outside of the build area and uses a hash of the full path to the build area as part of the name. Either that out of build area directory isn't being retained across builds, or the build area directory path is different every time the build runs (leading to new virtual environment each time).

    I recommend setting PIPENV_VENV_IN_PROJECT=1 before running pipenv. With this setting, your build will act more like you older build, with the virtualenv within the build directory tree. Like this:

    export PIPENV_VENV_IN_PROJECT=1
    pipenv install --dev
    pipenv run python run_tests.py
    

    (Also, no need to run pipenv install, pipenv install --dev will install the main packages along with the dev-only packages.)