Search code examples
pytorchcudapython-poetry

Poetry PyTorch dependency exclude cuda as I want to use the system cuda

I have these depencencies:

deepmultilingualpunctuation = "^1.0.1"

In the tree I see it requires torch

deepmultilingualpunctuation 1.0.1 A python package for deep multilingual punctuation prediction.
├── torch >=1.8.1
│   ├── filelock * 
│   ├── fsspec * 
│   ├── jinja2 * 
│   │   └── markupsafe >=2.0 
│   ├── networkx * 
│   ├── nvidia-cublas-cu12 12.1.3.1 
│   ├── nvidia-cuda-cupti-cu12 12.1.105 
│   ├── nvidia-cuda-nvrtc-cu12 12.1.105 
│   ├── nvidia-cuda-runtime-cu12 12.1.105

As you can see it includes the nvidia libs. These are already on my system, so I don't want to pull them in all my venvs

I tried adding a torch, extras=[cpu] dependency, but that does not help.

What is the proper solution to let Pytorch use the provided system libs?

Solution

  • I'm not sure how you tell it to use your system libraries for this.

    But for example if you want the CPU only version of torch you can do this:

    [tool.poetry.dependencies]
torch = { version = ">=2.0.0", source = "pytorch-cpu"}


[[tool.poetry.source]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cpu"
priority = "explicit"

    You can see the list of special builds here https://download.pytorch.org/whl/

    This link is also helpful https://pytorch.org/get-started/locally/