UserWarning: Applied workaround for CuDNN issue, install nvrtc.so

Question

I'm trainning a convolutional nueral network below.

def __init__(self, n_channels, n_classes):
    super().__init__()
    self.model = models.mobilenet_v3_large(pretrained=True)
    self.model.classifier[-1] = nn.Linear(1280, n_classes)
    self.model.features[0][0] = nn.Conv2d(
        n_channels,
        16,
        kernel_size=(3, 3),
        stride=(2, 2),
        padding=(1, 1),
        bias=False,
    )

there is only another thing in the warning that is

return F.conv2d(input, weight, bias, self.stride,

I don't know what is this warning and what can i do for it. if any information about the system is required, i can edit the question and add to it.

Answer 1

This seems to be an issue with the CUDA library in pytorch 2.0.x+cu118. There're a couple of possible fixes.

1. Downgrade to the previous CUDA library, cu117. E.g. with pip:

pip3 uninstall pytorch
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117) 

2. To continue to use cu118, create a symlink in the torch package library.

cd venv/python3.10/site-packages/torch/lib  # (or the path to dist-packages/torch/lib, depending on how you installed pytorch)

ln -s libnvrtc-*.so.11.2 libnvrtc.so

Alternatively, you should be able to ignore the warning; it shouldn't have any side effects or impact.

I found the above solutions in this GitHub issue, along with more in-depth discussions into the issue.