Best way to resolve conflicts between application packages

Question

I am developing a package that will be used by 3-party applications that I have no control over whatsoever.

I am wondering what is the best way to solve conflicts between packages.

For example:

Lets say a 3-party app wants to use my package called external-package.

├── Dockerfile
├── app.py
├── external-package
│   ├── Core
│   │   ├── ClassA.py
│   │   └── __init__.py
│   └── setup.py
└── requirements.txt

App.py:

from Core import ClassA

app = ClassA()
app.send()

ClassA:

import backoff
import requests

class ClassA:

    @backoff.on_exception(
        backoff.expo,
        requests.exceptions.RequestException,
        max_tries=35,
        backoff_log_level=20,
        max_time=2400,
        raise_on_giveup=True,
    )
    def send(self):
        requests.get('https://google.com')

Dockerfile:

FROM python:3.10-slim-buster

RUN apt-get update && apt-get install -y \
    automake \
    build-essential \
    libtool \
    wget \
    --no-install-recommends && \
    rm -rf /var/lib/apt/lists/*

WORKDIR /app

COPY ./external-package /app/external-package
RUN pip install --no-cache-dir /app/external-package

COPY requirements.txt /app/requirements.txt
RUN pip install --no-cache-dir -r /app/requirements.txt

COPY . /app

CMD ["python", "app.py"]

Running this container will outputs an error:

Traceback (most recent call last):
  File "/app/app.py", line 4, in <module>
    app.send()
  File "/usr/local/lib/python3.10/site-packages/backoff/_sync.py", line 87, in retry
    wait = _init_wait_gen(wait_gen, wait_gen_kwargs)
  File "/usr/local/lib/python3.10/site-packages/backoff/_common.py", line 23, in _init_wait_gen
    return wait_gen(**kwargs)
TypeError: expo() got an unexpected keyword argument 'backoff_log_level'

Because, the external-package must have backoff ==1.9.2, and the 3-party must have backoff ==2.2.1 but the container has backoff ==1.9.2.

Keep in mind this is an example only for backoff and I want a way to solve this issue for any library there is.

Answer 1

I have encountered this situation a few times while working with Python projects, and unfortunately, as of today there is no elegant way to install multiple versions of the same module in a the same Python environment.

What I'd do usually is to focus on what is within my control, which is the dependency of my own code (in your case, the external-package). Is it possible to not fix the backoff version for external-package (i.e. always default to latest version)? If yes, make the necessary code changes in external-package code to achieve that.

If not, then the next best alternative is probably to copy the source code of backoff==1.9.2 manually into external-package, which is a highly inelegant solution that may lead to issues in the future.