Search code examples
pythonpython-packagingpython-poetrypyproject.toml

Python version is higher in virtual environment than the one specified in pyproject.toml

I've currently set my pyproject.toml as follows

# ...
[tool.poetry.dependencies]
python = "^3.8.0"
pandas = "^2.0.0"
numpy = "^1.21.0"
requests = "^2.25.1"
# ...

I do then the following:

(base) PS C:\Users\project> poetry install
Installing dependencies from lock file
  - Installing numpy (1.24.4)
  - Installing python-dateutil (2.9.0.post0)
  - Installing pytz (2024.1)
  - Installing tzdata (2024.1)
  - Installing urllib3 (2.2.1)
  - Installing pandas (2.0.3)
  - Installing requests (2.32.3)

As far as I've understood the ^ should install the latest version of the max nest you provide in the pyproject.toml. So for example I was not expecting numpy to be higher than 1.21 (while it is instead 1.24.4).

The same applies for Python, which finally result in being 3.11:

(base) PS C:\Users\project> poetry shell
Spawning shell within C:\Users\AppData\Local\pypoetry\Cache\virtualenvs\project-mrBAX_Rz-py3.11
Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.

Install the latest PowerShell for new features and improvements! https://aka.ms/PSWindows

Loading personal and system profiles took 966ms.
(base) (project-py3.11) PS C:\Users\project>

The poetry.lock, which was not present before poetry install, has, among the others:

# ...
[[package]]
name = "numpy"
version = "1.24.4"
description = "Fundamental package for array computing in Python"
optional = false
python-versions = ">=3.8"
# ...

Why so? And how to configure it such that both the libraries and Python are not higher than the provided version (i.e. python = 3.8.XX, numpy=1.21.XX)

Solution

  • Note that placing upper bounds on the version constraints of the dependencies is quite frowned upon and can have bad effects in the Python community, in particular if your project is a library. Unless you know for sure that your project is not compatible with numpy>=1.22, then you definitely should not place any upper bound on the version constraint. This article explains the issue: https://iscinumpy.dev/post/bound-version-constraints/ (it is a long article, though, but I can not find a more concise article right now).

    You may want to use the "compatible release" notation (~=) with 3 segments.

    It is called "tilde requirements" in Poetry.

    For example:

    [tool.poetry.dependencies]
python = "~3.8.0"
numpy = "~1.21.0"

    This should be equivalent to the following:

    [tool.poetry.dependencies]
python = ">=3.8.0,<3.9.0"
numpy = ">=1.21.0,<1.22.0"