I'm using tensorflow==1.15.3 and I'm hitting a segmentation fault attempting int8 post-training quantization. The documentation for the 1.15 version of the TFLiteConverter can be found here.
I found a similar issue on github, but their solution to provide --add_postprocessing_op=true has not solved the segmentation fault.
I've debugged it using PDB and found exactly where it crashes. It never reaches my representative_dataset function. It faults when running CreateWrapperCPPFromBuffer(model_content):
> .../python3.6/site-packages/tensorflow_core/lite/python/optimize/calibrator.py(51)__init__()
-> .CreateWrapperCPPFromBuffer(model_content))
(Pdb) s
Fatal Python error: Segmentation fault
Current thread 0x00007ff40ee9f740 (most recent call first):
File ".../python3.6/site-packages/tensorflow_core/lite/python/optimize/calibrator.py", line 51 in __init__
File ".../python3.6/site-packages/tensorflow_core/lite/python/lite.py", line 236 in _calibrate_quantize_model
File ".../python3.6/site-packages/tensorflow_core/lite/python/lite.py", line 993 in convert
File ".../convert_model_to_tflite_int8.py", line 97 in <module>
File "<string>", line 1 in <module>
File "/usr/lib/python3.6/bdb.py", line 434 in run
File "/usr/lib/python3.6/pdb.py", line 1548 in _runscript
File "/usr/lib/python3.6/pdb.py", line 1667 in main
File "/usr/lib/python3.6/pdb.py", line 1694 in <module>
File "/usr/lib/python3.6/runpy.py", line 85 in _run_code
File "/usr/lib/python3.6/runpy.py", line 193 in _run_module_as_main
[1] 17668 segmentation fault (core dumped) python -m pdb convert_model_to_tflite_int8.py --add_postprocessing_op=true
Here is my conversion code:
converter = tf.lite.TFLiteConverter.from_frozen_graph(
graph_def_file=pb_model_path,
input_arrays=["device_0/input_node_name:1"],
output_arrays=["device_0/output_node_name"],
input_shapes={"device_0/input_node_name:1": [100, 16384]}
)
converter.allow_custom_ops = True
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.inference_input_type = tf.int8
converter.inference_output_type = tf.int8
def test():
pdb.set_trace()
print(' ! ! ! representative_dataset_gen ! ! ! ')
zeros = np.zeros(shape=(1, 100, 16384), dtype='int8')
ds = tf.data.Dataset.from_tensor_slices((zeros)).batch(1)
for input_value in ds.take(1):
yield [input_value]
converter.representative_dataset = test
pdb.set_trace()
tflite_model = converter.convert()
tflite_model_size = open(model_name, 'wb').write(tflite_model)
print('TFLite Model is %d bytes' % tflite_model_size)
FWIW my model conversion works for tf.float16 (not using representative_dataset there, though).
Upgrading my tf version to 2.3 solved the segmentation fault. My model code isn't compatible with tf==2.x yet, but luckily the conversion code is independent from that so the upgrade went smoothly.