Unable to utilize GPU for whisper AI. Only using CPU

Question

I am using WhisperAI from OpenAI to transcribe english and french audio. Git link here. I followed their instructions to install WhisperAI

The instance has a GPU, but torch.cuda.is_available() keeps returning false, so the model keeps using cpu

I initially programmed on a Mac M1 chip with cpu acceleration, but when I deployed it to the EC2 instance, I was expecting it to use GPU instead of CPU, but it is still using CPU.

How do I make the model use the GPU instead? Or how do I set it up to use GPU instead of CPU?

I am using tiny.en model for english transcription and tiny model for french transcription.

I am on an EC2 instance with a GPU, so I am trying to utilize GPU to make transcriptions faster. Here is my code:

import torch
import whisper

def transcribe(audio_file, language):
    if (language == 'english'):
        model_type = 'tiny.en'
    else:
        model_type = 'tiny'

    DEVICE = 'cuda' if torch.cuda.is_available() else 'cpu'
    model = whisper.load_model(model_type, device = DEVICE)
    transcription = model.transcribe(audio_file)
    return transcription

I am on python 3.9.16

My requirments.txt file looks like this:

ffmpeg-python==0.2.0
huggingface-hub==0.13.4
nltk==3.7
numba==0.56.4
numpy==1.23.5
openai-whisper @ git+https://github.com/openai/whisper.git@c09a7ae299c4c34c5839a76380ae407e7d785914
scikit-learn==1.1.3
scipy==1.10.1
sentence-transformers==2.2.2
sentencepiece==0.1.97
tensorstore==0.1.36
threadpoolctl==3.1.0
tiktoken==0.3.1
tokenizers==0.13.3
torch==2.0.1
torchaudio==2.0.2

Answer 1

Turns out i did not have nvidia drivers installed. Once i installed the drivers, i followed the answer on this question Cuda and OpenAI Whisper : enforcing GPU instead of CPU not working? and it was working for me