Search code examples
tensorflowonnxtensorrt

Accuracy drops in Tensorflow-Onnx-Tensorrt

I have a tensorflow trained model and tested at tensorflow with accuracy achieved 95%.

Tensorflow model is converted to ONNX and converted to TensorRT. TensorRT engine runs with 16-bit precision. In TensorRT, accuracy drops to 75%. Even with kTF32, accuracy is still 75%. Tested same images for both tests and same input size.

Where should I look at for this accuracy drop? The only difference is I do batch inference in TensorRT, but not in Tensorflow.

Images are read using OpenCV for both Tensorflow and TensorRT.

Can the accuracy drop in conversion from Tensorflow to ONNX?

Solution

  • Now I found the issue. Tensorflow did normalization to input image by multiplying 1/255.0. But in tensorrt normalization, it is 1- x/255.0. That is the issue. Now I have same accuracy.