YoloV5 way too less FPS (about 30fps), How can I improve it?

Question

I use yolov5s for game target detection, only 30fps.

Although the target inference time is only 10ms, I found that it takes about 15 seconds to complete the detection of 500 images, 500/15=33.33, which is about 33fps.

My gpu is 3070ti. This is the utilization rate during detection.

I have tried to change the version of pytorch and cuda, tried to use different weights, and tried to reduce the size of the detected image, but it didn't work.

Answer 1

Because your GPU not fully loaded, probably you preprocessing take match time. Evaluate real yolov5 performance on you hardware with code like this:

import time 

dummy_frame = torch.randn(32,3,416,416).cuda() #dummy data
model.eval()
model.cuda()

start = time.time()
for i in range(10):
  out = model(dummy_frame)
time_spent = time.time() - start
fps = 10*32 / time_spent   # batches * batch_size
print(f"{fps:.2f} fps") # 344 FPS on T4 in colab 

if fps on test significantly high then your current, try to improve your preprocessing pipeline