I have a working .tflite model (which takes 180x180 float greyscale image) as input, and returns 6 float sigmoid outputs. All works as-expected yielding expected results with test images.
I am trying to quantize .tflite model to uint8. Notice I am setting the input and output types to unit8 (edited for brevity):
converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_dataset_gen
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.uint8
converter.inference_output_type = tf.uint8
When I try to run inference on it with the following code (brevity):
print ("IMAGE_ARRAY type",img_array.dtype)
interpreter = tf.lite.Interpreter(model_path="bkgmodel_quant.tflite")
interpreter.resize_tensor_input(0, [1, 180, 180, 1])
interpreter.allocate_tensors()
print ("INPUT TENSOR",interpreter.get_input_details())
input = interpreter.tensor(interpreter.get_input_details()[0]["index"])
output = interpreter.tensor(interpreter.get_output_details()[0]["index"])
interpreter.set_tensor(0, img_array)
When I run I get the following error:
ValueError: Cannot set tensor: Got value of type UINT8 but expected type INT8 for input 0, name: input_2_int8
When I look at the types of the image data and input tensors, they inded are uint8:
IMAGE_ARRAY type uint8
INPUT TENSOR [{'name': 'input_2', 'index': 22, 'shape': array([ 1, 180, 180, 1], dtype=int32), 'shape_signature': array([ -1, 180, 180, 1], dtype=int32), 'dtype': <class 'numpy.uint8'>, 'quantization': (1.0, 0), 'quantization_parameters': {'scales': array([1.], dtype=float32), 'zero_points': array([0], dtype=int32), 'quantized_dimension': 0}, 'sparsity_parameters': {}}]
(look only at dtype in the input tensor, I assume).
So - indeed, the input tensor should be a uint8, and the image is a uint8 - yet I get this error.
As an experiment - I tried changing only my inference code to set the image data to int8. When I do, the error goes away and the inference runs, however my models predictions are all wrong.
(It is notable that the output values all add up to almost 256. I am assuming this means that the model is working correctly and yielding valid data - and I am also assuming that 8-bit sigmoids are expected to have a bit of a roundoff error where they don't add up to exactly 256??)
If I am doing something wrong, or is this a bug in Tensorflow-lite??
I upgraded from Tensorflow 2.3 to 2.5, and the problem appears to have been fixed.