We use a pipeline that first uses pip wheel to collect all the packages that are needed in the project and then it creates a docker image that calls to pip install on the collected wheels.
The issue I am encountering is that when calling pip wheel, pip is collecting 2 different versions of a package. This has started occurring once a new version of the package is available.
The project has a requirement for an internal library ecs-deployer==10.1.2 and that library has in turn a requirement in the form of: elb-listener>=3.2.1+25,<4
The relevant output of pip wheel with the verbose option says:
Collecting elb-listener>=3.2.1+25,<4
Created temporary directory: /tmp/pip-unpack-zr930807
File was already downloaded /home/user/path/dist/elb_listener-3.2.2+26-py3-none-any.whl
Added elb-listener>=3.2.1+25,<4 from https://internal-repository.com/path/elb_listener/3.2.2%2B26/elb_listener-3.2.2%2B26-py3-none-any.whl#md5=foo (from ecs-deployer==10.1.2->service==1.0.0) to build tracker '/tmp/pip-req-tracker-1tz9t5ls'
Removed elb-listener>=3.2.1+25,<4 from https://internal-repository.com/path/elb_listener/3.2.2%2B26/elb_listener-3.2.2%2B26-py3-none-any.whl#md5=blabla (from ecs-deployer==10.1.2->service==1.0.0) to build tracker '/tmp/pip-req-tracker-1tz9t5ls'
And also:
Collecting elb-listener>=3.2.1+25,<4
Created temporary directory: /tmp/pip-unpack-yfnxim_u
File was already downloaded /home/user/path/dist/elb_listener-3.2.3+27-py3-none-any.whl
Added elb-listener>=3.2.1+25,<4 from https://internal-repository.com/path/elb_listener/3.2.3%2B27/elb_listener-3.2.3%2B27-py3-none-any.whl#md5=bar (from ecs-deployer==10.1.2->service==1.0.0) to build tracker '/tmp/pip-req-tracker-1tz9t5ls'
Then when the pip install is called I get this:
ERROR: Cannot install elb-listener 3.2.2+26 (from /opt/elb_listener-3.2.2+26-py3-none-any.whl) and cad-aws-elb-listener-target-group-builder 3.2.3+27 (from /opt/elb_listener-3.2.3+27-py3-none-any.whl) because these package versions have conflicting dependencies.
The conflict is caused by:
The user requested elb-listener 3.2.2+26 (from /opt/elb_listener-3.2.2+26-py3-none-any.whl)
The user requested elb-listener 3.2.3+27 (from /opt/elb_listener-3.2.3+27-py3-none-any.whl)
We use pip 20.2.3 with the option --use-feature=2020-resolver
Is it normal that pip wheel collects several versions of the same package?
If so, can I indicate in any way to either pip wheel to only collect one of the versions or to pip install to only use the latest version?
If not, is there any way to solve this problem? I guess changing the requirement to elb-listener>=3.2.1+27,<4 would solve it, but we don't have direct access to that library and it would take a while for other team to change it.
As per @sinoroc comment, upgrading the python to 3.10 and pip version to 21.2.4 solved this particular issue.