Search code examples
pythongitdockerpippython-wheel

pip - How to use pre-built wheel instead of pulling git again to avoid conflict?

Context
In my Django project (based on Django cookiecutter) I use django-graphql-auth which depends on django-graphql-jwt.
I forked django-graphql-jwt to make some changes so then also forked django-graphql-auth to update its dependency to my django-graphql-jwt fork:

# django-graphql-auth setup.py
install_requires=[
    "django-graphql-jwt @ git+<git_url>#egg=django_graphql_jwt",
    ...,
]

This works as expected with pip install -r requirements.txt.

Problem
In Docker, when I build wheels in one stage and install them in another, the django-graphql-jwt git is pulled twice (on build and on install) and a conflict occurs.
Cookiecutter Django provides a Dockerfile (found here) which is split into multiple stages:

  1. Wheels are built for all dependencies. This is when both -auth and -jwt git are cloned and built.
> pip wheel --wheel-dir /wheels/ -r local.txt
  1. Wheels are copied from the previous stage and installed. Here, built wheels should be used (no cloning of git).
> pip install --no-cache-dir --no-index --find-links=/wheels/ /wheels/*
...
Processing /wheels/django_graphql_auth-0.3.16-py2.py3-none-any.whl
Processing /wheels/django_graphql_jwt-0.3.4-py3-none-any.whl
...
Collecting django-graphql-jwt@ git+<git url>
    Cloning ...
...
ERROR: Cannot install django-graphql-auth==0.3.16 and django-graphql-jwt 0.3.4 (from /wheels/django_graphql_jwt-0.3.4-py3-none-any.whl) because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested django-graphql-jwt 0.3.4 (from /wheels/django_graphql_jwt-0.3.4-py3-none-any.whl)
    django-graphql-auth 0.3.16 depends on django-graphql-jwt (unavailable)

As you can see the existing -jwt wheel is processed but afterwards, its git is cloned. These two seem to result in a conflict. If I add a version in setup.py (django-graphql-jwt>=0.3.4) it already fails on the build step.

How can I match the -auth dependency to the already built -jwt wheel?

Solution

  • Assuming all required dependencies were built in the first step (with pip wheel), you could ignore dependencies in the installation step by adding the --no-deps option to pip install:

    pip install --no-cache-dir --no-index --no-deps --find-links=/wheels/ /wheels/*