Search code examples
pythonopenvino

OpenVINO failed to set Blob with FP16 precision not corresponding to user input precision

I commanded --data_type FP16 to confirm I could use FP16 precision when I was generating IR format files.

python mo_tf.py --saved_model_dir C:\Users\Hsien\Desktop\OCT\saved_model --input_shape [1,256,256,3] --data_type FP16

But I got an error message when I tried to execute my inference engine code with FP16 precision and device assignation was MYRIAD or GPU.

Traceback (most recent call last):
  File "C:\Users\Hsien\Desktop\OCT\test.py", line 27, in <module>
    inference_request.set_blob(blob_name=input_blob_name, blob=input_blob)
  File "ie_api.pyx", line 1438, in openvino.inference_engine.ie_api.InferRequest.set_blob
RuntimeError: [ PARAMETER_MISMATCH ] Failed to set Blob with precision not corresponding to user input precision

Here comes my inference engine code

from openvino.inference_engine import IECore, Blob, TensorDesc
import cv2
import numpy as np

IMG_PATH = r"C:\Users\Hsien\Desktop\OCT\input_crop.jpg"
XML_PATH = r"C:\Users\Hsien\Desktop\OCT\saved_model\saved_model.xml"
BIN_PATH = r"C:\Users\Hsien\Desktop\OCT\saved_model\saved_model.bin"

ie_core_handler = IECore()
network = ie_core_handler.read_network(model=XML_PATH, weights=BIN_PATH)
executable_network = ie_core_handler.load_network(network, device_name='MYRIAD', num_requests=1)
inference_request = executable_network.requests[0]

image = cv2.imread(IMG_PATH)
re_img = cv2.resize(src=image, dsize=(256, 256))
input_data = np.expand_dims(np.transpose(re_img, (2, 0, 1)), 0).astype(np.float16)
tensor_description = TensorDesc(precision="FP16", dims=(1, 3, 256, 256), layout='NCHW')
input_blob = Blob(tensor_description, input_data)

input_blob_name = next(iter(inference_request.input_blobs))
inference_request.set_blob(blob_name=input_blob_name, blob=input_blob)
inference_request.infer()
output_blob_name = next(iter(inference_request.output_blobs))
output = inference_request.output_blobs[output_blob_name].buffer
print(output)

And it can work with FP32 precision and device assignation is CPU, which I only change the following three lines.

executable_network = ie_core_handler.load_network(network, device_name='CPU', num_requests=1)
input_data = np.expand_dims(np.transpose(re_img, (2, 0, 1)), 0).astype(np.float32)
tensor_description = TensorDesc(precision="FP32", dims=(1, 3, 256, 256), layout='NCHW')

My environment is Windows 11 with openvino_2021.4.752 version.

And my devices are Intel® Core™ i7-11700K with Intel® UHD Graphics 750 and Intel® Neural Compute Stick 2.

Solution

  • The error you encountered: Failed to set Blob with precision not corresponding to user input precision was due to the input precision is set to FP32 in default. Refer to Step 2. Configure Input and Output of the Model, in Integrate Inference Engine.

    Please add the following code before executable_network:

    # Get names of the input blob
input_blob_name = next(iter(network.input_info))
# Set input precision
network.input_info[input_blob_name].precision = 'FP16'