Search code examples
pythonpython-poetry

poetry built package with os-specific dependencies

I've built a package using poetry. The package uses geopandas (which means in turn GDAL, Fiona as well).

GDAL is a notoriously pain to install on windows and I ended with a pyproject.toml file using an os-specific dependency description, ie :

[tool.poetry.dependencies]
python = "^3.9"
dukpy = [
    {platform = 'windows', url = "https://download.lfd.uci.edu/pythonlibs/archived/dukpy-0.2.3-cp39-cp39-win_amd64.whl"},
    {platform = 'linux', version = "~0.2"},
    ]    
Rtree = "^1.0.0"
GDAL = [
    {platform = 'windows', url = "https://download.lfd.uci.edu/pythonlibs/archived/GDAL-3.4.3-cp39-cp39-win_amd64.whl"},
    {platform = 'linux', version = "~3.4"},
    ]
Fiona = [
    {markers = "sys_platform == 'win64'", url = "https://download.lfd.uci.edu/pythonlibs/archived/Fiona-1.8.21-cp39-cp39-win_amd64.whl"},
    {markers = "sys_platform == 'linux'", version = "~1.8"}
    ]
geopandas = "^0.11.1"

The package was built using poetry (on a linux server) and runs well. When I try to install it (using pip) on a windows machine, python tries to install all dependencies from pypi (right now for instance, it downloads Fiona-1.8.21.tar.gz instead of downloading the pre-compiled whl).

Question : is this possible to retain the os-specific dependencies after building the wheel ?

Note 1 : I'm aware how to perform the installation on windows : I can install manually all dependecies from url first and my built package afterwards. But I hope there is an easier way to perform this task.

Note 2 : I'm aware the lfd "repo" is archived, but these are still working and up to date - for now...

EDIT

Extract from the medata file:

Requires-Python: >=3.9,<4.0
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Requires-Dist: Fiona (>=1.8,<1.9); sys_platform == "linux"
Requires-Dist: Fiona @ https://download.lfd.uci.edu/pythonlibs/archived/Fiona-1.8.21-cp39-cp39-win_amd64.whl ; sys_platform == "win64"
Requires-Dist: GDAL (>=3.4,<3.5); sys_platform == "linux"
Requires-Dist: GDAL @ https://download.lfd.uci.edu/pythonlibs/archived/GDAL-3.4.3-cp39-cp39-win_amd64.whl ; sys_platform == "windows"
Requires-Dist: Rtree (>=1.0.0,<2.0.0)
Requires-Dist: geopandas (>=0.11.1,<0.12.0)

It keeps the os-specific description indeed...

Solution

  • Right – the thing is that sys.platform(), i.e. the sys_platform marker's value, is always win32, never windows or win64.

    Those windows or win64 dependencies will never get selected, since they don't match.

    You can find the known environment markers here; platform_system == "Windows" could apparently also work.

    Try with

    [tool.poetry.dependencies]
python = "^3.9"
dukpy = [
    {platform = 'win32', url = "https://download.lfd.uci.edu/pythonlibs/archived/dukpy-0.2.3-cp39-cp39-win_amd64.whl"},
    {platform = 'linux', version = "~0.2"},
    ]    
Rtree = "^1.0.0"
GDAL = [
    {platform = 'win32', url = "https://download.lfd.uci.edu/pythonlibs/archived/GDAL-3.4.3-cp39-cp39-win_amd64.whl"},
    {platform = 'linux', version = "~3.4"},
    ]
Fiona = [
    {platform = "win32", url = "https://download.lfd.uci.edu/pythonlibs/archived/Fiona-1.8.21-cp39-cp39-win_amd64.whl"},
    {platform = "linux", version = "~1.8"}
    ]
geopandas = "^0.11.1"