How to handle diamond dependency in Python?

Question

My project has two dependencies, and each of them transitively depends on a different major version of protobuf.

The general situation is well described

• in an article Google Best Practices for Java Libraries: What is a diamond dependency conflict?
• and also Software Engineering at Google; Dependency Management: Conflicting Requirements and Diamond Dependencies.

The specific problem in my project manifests with this error message from Poetry.

kfp (1.8.21) depends on protobuf (>=3.13.0,<4)
and robotframework-browser (18.0.0) depends on protobuf (4.25.1)

These packages operate completely independently. They do not share data. If each was able to use its own protobuf version, things would work out. Is this possible in Python?

Answer 1

Unfortunately not easily at all. A single Python environment can't have two copies of a package by the same name.

Your major options are to

1. Find a combination of versions of kfp and robotframework-browser that have compatible protobuf requirements (kfp~=1.8 is pretty old though, you might consider upgrading!)
2. See if you can force Poetry to install protobuf~=4 anyway, and see if kfp works with it
3. See if you can force Poetry to install protobuf~=3 anyway, and see if robotframework-browser works with it

Beyond that, if you can't upgrade kfp, you could fork and patch kfp to depend on and use protobuf>=4...