I'm trying to convert a keras model which I trained and fine tuned with quantization aware training tutorial on their official website to a int tflite model. I am able to follow their steps until I have to convert the model to tflite format. Then it gives me this output:
`Traceback (most recent call last):
File "/home/student1/venv_kvant/lib/python3.6/site-packages/tensorflow/lite/python/convert.py", line 185, in toco_convert_protos
enable_mlir_converter)
File "/home/student1/venv_kvant/lib/python3.6/site-packages/tensorflow/lite/python/wrap_toco.py", line 38, in wrapped_toco_convert
enable_mlir_converter)
Exception: /home/student1/venv_kvant/lib/python3.6/site-packages/tensorflow/python/keras/layers/ops/core.py:56:1: error: 'std.constant' op requires attribute's type ('tensor<48x64xf32>') to match op's return type ('tensor<*xf32>')
outputs = standard_ops.tensordot(inputs, kernel, [[rank - 1], [0]])
^
/home/student1/venv_kvant/lib/python3.6/site-packages/tensorflow/python/keras/layers/core.py:1194:1: note: called from
dtype=self._compute_dtype_object)
^
/home/student1/venv_kvant/lib/python3.6/site-packages/tensorflow_model_optimization/python/core/quantization/keras/quantize_wrapper.py:162:1: note: called from
outputs = self.layer.call(inputs)
^
/home/student1/venv_kvant/lib/python3.6/site-packages/tensorflow/python/autograph/impl/api.py:302:1: note: called from
return func(*args, **kwargs)
^
/home/student1/venv_kvant/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py:961:1: note: called from
outputs = call_fn(inputs, *args, **kwargs)
^
/home/student1/venv_kvant/lib/python3.6/site-packages/tensorflow/python/keras/engine/functional.py:507:1: note: called from
outputs = node.layer(*args, **kwargs)
^
/home/student1/venv_kvant/lib/python3.6/site-packages/tensorflow/python/keras/engine/functional.py:385:1: note: called from
inputs, training=training, mask=mask)
^
/home/student1/venv_kvant/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py:961:1: note: called from
outputs = call_fn(inputs, *args, **kwargs)
^
/home/student1/venv_kvant/lib/python3.6/site-packages/tensorflow/python/keras/saving/saving_utils.py:132:1: note: called from
outputs = model(inputs, training=False)
^
/home/student1/venv_kvant/lib/python3.6/site-packages/tensorflow/python/eager/def_function.py:600:1: note: called from
return weak_wrapped_fn().__wrapped__(*args, **kwds)
^
/home/student1/venv_kvant/lib/python3.6/site-packages/tensorflow/python/keras/layers/ops/core.py:56:1: note: see current operation: %cst_8 = "std.constant"() {value = dense<"0x38211AB .. A3E"> : tensor<48x64xf32>} : () -> tensor<*xf32>
outputs = standard_ops.tensordot(inputs, kernel, [[rank - 1], [0]])
^
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/student1/kvantizacija/tensorflow_example.py", line 58, in <module>
tflite_model_quant = converter.convert()
File "/home/student1/venv_kvant/lib/python3.6/site-packages/tensorflow/lite/python/lite.py", line 778, in convert
self).convert(graph_def, input_tensors, output_tensors)
File "/home/student1/venv_kvant/lib/python3.6/site-packages/tensorflow/lite/python/lite.py", line 595, in convert
**converter_kwargs)
File "/home/student1/venv_kvant/lib/python3.6/site-packages/tensorflow/lite/python/convert.py", line 560, in toco_convert_impl
enable_mlir_converter=enable_mlir_converter)
File "/home/student1/venv_kvant/lib/python3.6/site-packages/tensorflow/lite/python/convert.py", line 188, in toco_convert_protos
raise ConverterError(str(e))
tensorflow.lite.python.convert.ConverterError: /home/student1/venv_kvant/lib/python3.6/site-packages/tensorflow/python/keras/layers/ops/core.py:56:1: error: 'std.constant' op requires attribute's type ('tensor<48x64xf32>') to match op's return type ('tensor<*xf32>')
outputs = standard_ops.tensordot(inputs, kernel, [[rank - 1], [0]])
^
/home/student1/venv_kvant/lib/python3.6/site-packages/tensorflow/python/keras/layers/core.py:1194:1: note: called from
dtype=self._compute_dtype_object)
^
/home/student1/venv_kvant/lib/python3.6/site-packages/tensorflow_model_optimization/python/core/quantization/keras/quantize_wrapper.py:162:1: note: called from
outputs = self.layer.call(inputs)
^
/home/student1/venv_kvant/lib/python3.6/site-packages/tensorflow/python/autograph/impl/api.py:302:1: note: called from
return func(*args, **kwargs)
^
/home/student1/venv_kvant/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py:961:1: note: called from
outputs = call_fn(inputs, *args, **kwargs)
^
/home/student1/venv_kvant/lib/python3.6/site-packages/tensorflow/python/keras/engine/functional.py:507:1: note: called from
outputs = node.layer(*args, **kwargs)
^
/home/student1/venv_kvant/lib/python3.6/site-packages/tensorflow/python/keras/engine/functional.py:385:1: note: called from
inputs, training=training, mask=mask)
^
/home/student1/venv_kvant/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py:961:1: note: called from
outputs = call_fn(inputs, *args, **kwargs)
^
/home/student1/venv_kvant/lib/python3.6/site-packages/tensorflow/python/keras/saving/saving_utils.py:132:1: note: called from
outputs = model(inputs, training=False)
^
/home/student1/venv_kvant/lib/python3.6/site-packages/tensorflow/python/eager/def_function.py:600:1: note: called from
return weak_wrapped_fn().__wrapped__(*args, **kwds)
^
/home/student1/venv_kvant/lib/python3.6/site-packages/tensorflow/python/keras/layers/ops/core.py:56:1: note: see current operation: %cst_8 = "std.constant"() {value = dense<"0x38211ABEE ... 6D3DE88D49BE40211A3E"> : tensor<48x64xf32>} : () -> tensor<*xf32>
outputs = standard_ops.tensordot(inputs, kernel, [[rank - 1], [0]])
^
Process finished with exit code 1
If I remove the flag for optimization it gives me a tflite model but it doesn't give me the int8 model I need. I can successfully post train quantize the same model I fine tune with quant aware training but for some reason when I wrap the model in Quantize wrappers and try to convert it it doesn't work. I'm using the latest nightly version and tried running the script both with and without the access to GPU. `
If you need any more information feel free to ask. One more this the model is are 4 CNN + Max_pool blocks with some Dense layers at the end. If needed I can provide the vizualization of the model.
PS. Here is the summary:
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input (InputLayer) [(None, 48, 48, 3)] 0
__________________________________________________________________________________________________
quantize_layer (QuantizeLayer) (None, 48, 48, 3) 3 input[0][0]
__________________________________________________________________________________________________
quant_conv_1 (QuantizeWrapper) (None, 46, 46, 16) 483 quantize_layer[0][0]
__________________________________________________________________________________________________
quant_relu_1 (QuantizeWrapper) (None, 46, 46, 16) 3 quant_conv_1[0][0]
__________________________________________________________________________________________________
quant_pool_1 (QuantizeWrapper) (None, 22, 22, 16) 1 quant_relu_1[0][0]
__________________________________________________________________________________________________
quant_conv_2 (QuantizeWrapper) (None, 20, 20, 32) 4707 quant_pool_1[0][0]
__________________________________________________________________________________________________
quant_relu_2 (QuantizeWrapper) (None, 20, 20, 32) 3 quant_conv_2[0][0]
__________________________________________________________________________________________________
quant_pool_2 (QuantizeWrapper) (None, 9, 9, 32) 1 quant_relu_2[0][0]
__________________________________________________________________________________________________
quant_conv_3 (QuantizeWrapper) (None, 7, 7, 32) 9315 quant_pool_2[0][0]
__________________________________________________________________________________________________
quant_relu_3 (QuantizeWrapper) (None, 7, 7, 32) 3 quant_conv_3[0][0]
__________________________________________________________________________________________________
quant_pool_3 (QuantizeWrapper) (None, 3, 3, 32) 1 quant_relu_3[0][0]
__________________________________________________________________________________________________
quant_conv_4 (QuantizeWrapper) (None, 2, 2, 64) 8387 quant_pool_3[0][0]
__________________________________________________________________________________________________
quant_pool_4 (QuantizeWrapper) (None, 1, 1, 64) 1 quant_conv_4[0][0]
__________________________________________________________________________________________________
quant_relu_4 (QuantizeWrapper) (None, 1, 1, 64) 3 quant_pool_4[0][0]
__________________________________________________________________________________________________
quant_fc_yaw (QuantizeWrapper) (None, 1, 1, 48) 3125 quant_relu_4[0][0]
__________________________________________________________________________________________________
quant_fc_pitch (QuantizeWrapper (None, 1, 1, 48) 3125 quant_relu_4[0][0]
__________________________________________________________________________________________________
quant_fc_roll (QuantizeWrapper) (None, 1, 1, 48) 3125 quant_relu_4[0][0]
__________________________________________________________________________________________________
quant_relu_yaw (QuantizeWrapper (None, 1, 1, 48) 3 quant_fc_yaw[0][0]
__________________________________________________________________________________________________
quant_relu_pitch (QuantizeWrapp (None, 1, 1, 48) 3 quant_fc_pitch[0][0]
__________________________________________________________________________________________________
quant_relu_roll (QuantizeWrappe (None, 1, 1, 48) 3 quant_fc_roll[0][0]
__________________________________________________________________________________________________
quant_flatten_yaw (QuantizeWrap (None, 48) 1 quant_relu_yaw[0][0]
__________________________________________________________________________________________________
quant_flatten_pitch (QuantizeWr (None, 48) 1 quant_relu_pitch[0][0]
__________________________________________________________________________________________________
quant_flatten_roll (QuantizeWra (None, 48) 1 quant_relu_roll[0][0]
__________________________________________________________________________________________________
quant_output_yaw (QuantizeWrapp (None, 61) 2994 quant_flatten_yaw[0][0]
__________________________________________________________________________________________________
quant_output_pitch (QuantizeWra (None, 61) 2994 quant_flatten_pitch[0][0]
__________________________________________________________________________________________________
quant_output_roll (QuantizeWrap (None, 61) 2994 quant_flatten_roll[0][0]
__________________________________________________________________________________________________
quant_yaw (QuantizeWrapper) (None, 1) 54 quant_flatten_yaw[0][0]
__________________________________________________________________________________________________
quant_pitch (QuantizeWrapper) (None, 1) 54 quant_flatten_pitch[0][0]
__________________________________________________________________________________________________
quant_roll (QuantizeWrapper) (None, 1) 54 quant_flatten_roll[0][0]
==================================================================================================
Total params: 41,442
Trainable params: 41,066
Non-trainable params: 376
Greetings this is closed due to finding another way to resolve the problem. The problem was that there was a sequence of layers: -> Dense -> Flatten -> Dense and because of it this error was occurring. For the time being the solution I'm using is to switch places of Flatten layer and 1st Dense layer. If anybody know how to resolve using the original sequence please let me know.