How can I query to REST API runs on tensorflow_model_server?

Question

I'd tried to run simple TensorFlow estimators example: official Iris classification problem and saved model using this code implemented by this tutorial.

TensorFlow provides a command line tool to inspect the exported model like the following:

$ saved_model_cli show --dir export/1550568903/ \
> --tag_set serve --signature_def serving_default
The given SavedModel SignatureDef contains the following input(s):
  inputs['inputs'] tensor_info:
      dtype: DT_STRING
      shape: (-1)
      name: input_example_tensor:0
The given SavedModel SignatureDef contains the following output(s):
  outputs['classes'] tensor_info:
      dtype: DT_STRING
      shape: (-1, 3)
      name: dnn/head/Tile:0
  outputs['scores'] tensor_info:
      dtype: DT_FLOAT
      shape: (-1, 3)
      name: dnn/head/predictions/probabilities:0
Method name is: tensorflow/serving/classify

I'd installed tensorflow-model-server and started a model server that supports REST API:

$ apt-get install tensorflow-model-server
$ tensorflow_model_server --port=9000 --rest_api_port=9001 --model_base_path=/home/deploy/export
...
2019-02-22 11:36:44.989600: I tensorflow_serving/model_servers/server.cc:302] Exporting HTTP/REST API at:localhost:9001 ...

and then I called the REST API below like:

$ curl -d '{"inputs":[{"SepalLength":[5.1],"SepalWidth":[3.3],"PetalLength":[1.7],"PetalWidth":[0.5]}]}' \
   -X POST http://localhost:9001/v1/models/default:predict

{ "error": "JSON Value: {\n \"SepalLength\": [\n 5.1\n ],\n \"SepalWidth\": [\n 3.3\n ],\n \"PetalLength\": [\n 1.7\n ],\n \"PetalWidth\": [\n 0.5\n ]\n} not formatted correctly for base64 data" }

An error occurred called "not formatted correctly for base64 data." So, I encoded inputs as follows:

$ curl -d '{"inputs": [{"b64": "W3siU2VwYWxMZW5ndGgiOls1LjFdLCJTZXBhbFdpZHRoIjpbMy4zXSwiUGV0YWxMZW5ndGgiOlsxLjddLCJQZXRhbFdpZHRoIjpbMC41XX1d"}]}' \
   -X POST http://localhost:9001/v1/models/default:predict

However, there's still the following error:

{ "error": "Could not parse example input, value: \'[{\"SepalLength\":[5.1],\"SepalWidth\":[3.3],\"PetalLength\":[1.7],\"PetalWidth\":[0.5]}]\'\n\t [[{{node ParseExample/ParseExample}} = ParseExample[Ndense=4, Nsparse=0, Tdense=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], _output_shapes=[[?,1], [?,1], [?,1], [?,1]], dense_shapes=[[1], [1], [1], [1]], sparse_types=[], _device=\"/job:localhost/replica:0/task:0/device:CPU:0\"](_arg_input_example_tensor_0_0, ParseExample/ParseExample/names, ParseExample/ParseExample/dense_keys_0, ParseExample/ParseExample/dense_keys_1, ParseExample/ParseExample/dense_keys_2, ParseExample/ParseExample/dense_keys_3, ParseExample/Const, ParseExample/Const, ParseExample/Const, ParseExample/Const)]]" }

What am I doing wrong? How can I call the REST API without error?

Answer 1

I have tried to reproduce your error and I got the similar error for Curl Predict.

But when I have used Classify, I got the output.

Code is shown below:

curl -d '{"examples":[{"SepalLength":[5.1],"SepalWidth":[3.3],"PetalLength":[1.7],"PetalWidth":[0.5]}]}' -X POST http://localhost:8501/v1/models/export:classify

Output is:

 {"results": [[["0", 0.998091], ["1", 0.00190929], ["2", 1.46236e-08]]]}