Search code examples
tensorflowamazon-sagemaker

How to infer a tensorflow pre trained deployed model with an image?

I have an already trained a TensorFlow model outside of SageMaker.

I am trying to focus on deployment/inference but I am facing issues with inference.

For deployment I did this:

from sagemaker.tensorflow.serving import TensorFlowModel
instance_type = 'ml.c5.xlarge' 

model = TensorFlowModel(
    model_data=model_data,
    name= 'tfmodel1',
    framework_version="2.2",
    role=role, 
    source_dir='code',
)

predictor = model.deploy(endpoint_name='test', 
                                       initial_instance_count=1, 
                                       tags=tags,
                                       instance_type=instance_type)

When I tried to infer the model I did this:

import PIL
from PIL import Image
import numpy as np
import json
import boto3

image = PIL.Image.open('img_test.jpg')
client = boto3.client('sagemaker-runtime')
batch_size = 1
image = np.asarray(image.resize((512, 512)))
image = np.concatenate([image[np.newaxis, :, :]] * batch_size)
body = json.dumps({"instances": image.tolist()})

ioc_predictor_endpoint_name = "test"
content_type = 'application/x-image'   
ioc_response = client.invoke_endpoint(
    EndpointName=ioc_predictor_endpoint_name,
    Body=body,
    ContentType=content_type
 )

But I have this error:

ModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (415) from primary with message "{"error": "Unsupported Media Type: application/x-image"}".

I also tried:

from sagemaker.predictor import Predictor

predictor = Predictor(ioc_predictor_endpoint_name)
inference_response = predictor.predict(data=body)
print(inference_response)

And have this error:

ModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (415) from primary with message "{"error": "Unsupported Media Type: application/octet-stream"}".

What can I do ? I don't know if I missed something

Solution

  • Have you tested this model locally? How does inference work with your TF model locally? This should show you how the input needs to be formatted for inference with that model in specific. Application/x-image data format should be fine. Do you have a custom inference script? Check out this link here for adding an inference script with will let you control pre/post processing and you can log each line to capture the error: https://github.com/aws/sagemaker-tensorflow-serving-container.