Search code examples
pytorch

pytorch unable to run inference with GPU


I'm developing a project based on yolov7, but I started facing this error where torch recognizes my GPU but torchvision throws an Not Implemented Error.

This is the error

NotImplementedError: Could not run 'torchvision::nms' with arguments from the 'CUDA' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook employee using PyTorch on mobile, please visit https://fburl.com/ptmfixes for possible resolutions. 'torchvision::nms' is only available for these backends: [CPU, QuantizedCPU, BackendSelect, Python, FuncTorchDynamicLayerBackMode, Functionalize, Named, Conjugate, Negative, ZeroTensor, ADInplaceOrView, AutogradOther, AutogradCPU, AutogradCUDA, AutogradXLA, AutogradMPS, AutogradXPU, AutogradHPU, AutogradLazy, Tracer, AutocastCPU, AutocastCUDA, FuncTorchBatched, FuncTorchVmapMode, Batched, VmapMode, FuncTorchGradWrapper, PythonTLSSnapshot, FuncTorchDynamicLayerFrontMode, PythonDispatcher].

I tried installing torchvision with cuda built-in but that gave me the same error, also tried reinstalling pytorch , that didn't work either


Solution

  • the version of torch vision installed in my env was not equipped with cuda as it was a common install with pip with pip install torchvision whereas for torchvision to function with cuda it has to be equipped with cuda in-order for it to function with an Nvidia GPU to do so install torch with the following command conda install pytorch torchvision torchaudio pytorch-cuda={CUDA version} -c pytorch -c nvidia