How can I return a NumPy array using FastAPI?

I have a TensorFlow Keras deep learning model in the form of an h5 file.

How can I upload an image and return a NumPy array in FastAPI?

import numpy as np
import cv2
from fastapi import FastAPI, File, UploadFile
import numpy as np
from tensorflow.keras.models import load_model
import tensorflow as tf

model=load_model("complete_model.h5")
app = FastAPI()

def prepare(image):
    IMG_SIZE = 224
    new_array = cv2.resize(image, (IMG_SIZE, IMG_SIZE))
    return new_array.reshape(-1, IMG_SIZE,IMG_SIZE,3)

@app.post("/")
async def root(file: UploadFile = File(...)):
    global model
    content = await file.read()
    nparr = np.fromstring(content, np.uint8)
    img = cv2.imdecode(nparr, cv2.IMREAD_COLOR).astype(np.float32)
    prediction = model.predict(prepare(img))
    return prediction

When uploading the image using Swagger UI, I get the following error:

line 137, in jsonable_encoder
data = dict(obj)
TypeError: 'numpy.float32' object is not iterable

Working code without FastAPI:

import numpy as np
import numpy as np
from tensorflow.keras.models import load_model
import tensorflow as tf
import cv2

model=load_model("complete_model.h5")

def prepare(image):
    IMG_SIZE = 224
    new_array = cv2.resize(image, (IMG_SIZE, IMG_SIZE))
    return new_array.reshape(-1, IMG_SIZE,IMG_SIZE,3)

img = cv2.imread("./test.jpeg").astype(np.float32)
prediction = model.predict(prepare(img))
print(prediction)

Result in the terminal:

[[0.25442022 0.74557984]]

How can I get the same result while using FastAPI?

Solution

The error is thrown when returning the response (i.e., prediction in your case) from your endpoint. It looks like FastAPI is trying to convert the NumPy array into a dict, using the jsonable_encoder, which is used internally by FastAPI when returning a value from an endpoint, and which seems to call Python's vars() method, as shown in the error you provided here (have a look at the discussion here, as well as the documentation). Thus, what you could do is to convert the NumPy array into a Python list, then serialize it into a JSON string and return it:

return json.dumps(prediction.tolist())

Note that instead of returning the JSON string in the way it is shown above, which would casue FastAPI to serialize it again behind the scenes, you might consider returning a custom Response directly, as demonstrated here, as well as here and here.

On Swagger UI /docs, you should still be able to see the expected result. However, if you needed to convert it back to a NumPy array, you could parse the JSON string in Python, as shown below.

arr = np.asarray(json.loads(resp.json()))  # resp.json() if using Python requests

If you would like to return the NumPy array as raw bytes and display the image in the browser or download it, have a look at this answer.