poetry can't find version of dependency even though it exists

Question

When bumping my python version from 3.7 to 3.8 in poetry, reinstalling all the dependencies fail with a version of the following:

ERROR: No matching distribution found for...

The distribution for that version is available at pypa, and is often the most recent version.

Simply removing the offending package doesn't fix the issue, as poetry will likely fail with other packages. After some investigation, it appears that somehow poetry isn't using pip3 to install underneath, but is instead using pip2.7.

Indeed this is supported by a deprecation alert, and the error is always reproducible if I attempt to install the same version with pip (globally or otherwise) and not pip3.

This issue is frustrating and deleting the venv alone doesn't seem to help. How can I resolve this dependency issue that shouldn't exist in the first place?

Answer 1

There are two issues here which feed into each other.

1. poetry seems to consistently botch the upgrade of a venv when you modify the python versions. According to finswimmer, the upgrade should create a new virtual env for the new python version, however this process can fail when poetry uses the wrong pip version or loses track of which virtual env it's using.

2. poetry uses whatever pip is no questions asked - with no way to override and force usage of pip3.

Here are the distilled steps I used to solve this issue

1. delete the virtual env ( sometimes poetry loses track of the venv/thinks it's already activated. Best to clear the slate )

rm -rf `poetry env list --full-path`

2. create a new virtual env ( the command should fail, but the venv will be created )

poetry install

3. manually activate the virtual env

source "$(poetry env list --full-path | tail -1 | sed 's/.\{12\}$//')/bin/activate"

4. poetry install within the virtual env ( this ensures poetry is using the correct version of pip )

poetry install