Tensorflow Serve SignatureDefs for classifier

Question

I trained a BERT text classifier following these steps, with own texts and some modifications:

https://www.tensorflow.org/tutorials/text/classify_text_with_bert

To export the model and run it with Tensorflow Serve works well:

https://www.tensorflow.org/tfx/serving/docker

Unfortunately, I can not really figure out how to define the SignatureDefs for a classifier, so that the classifier endpoint for Tensorflow Serve is defined. The :predict endpoint works well and seems to be the default signature.

Obviously, I have to define the signatures, when I save the model. Since the documentation is not very exhaustive on this topic, I am not sure how to define the classifier signature.

https://www.tensorflow.org/tfx/serving/signature_defs

In the above example for BERT, the serving_results just define the reloaded_model with tf.constant(examples) and instantiates it with serving_results = tf.sigmoid(serving_results['classifier')].

So, I assume I have to give the activation function and the classifier signature as arguments, when calling the model.save method.

predictendpoint works, classify endpoint gives the error:

{"error": "No classification inputs found in SignatureDef: inputs {\n key: \"text\"\n value {\n name: \"serving_default_text:0\"\n dtype: DT_STRING\n tensor_shape {\n dim {\n size: -1\n }\n }\n }\n}\noutputs {\n key: \"classifier\"\n value {\n name: \"StatefulPartitionedCall_2:0\"\n dtype: DT_FLOAT\n tensor_shape {\n dim {\n size: -1\n }\n dim {\n size: 1\n }\n }\n }\n}\nmethod_name: \"tensorflow/serving/predict\"\n"}

I would be grateful for any hints.

Answer 1

The right documentation for defining the Signatures can be found here.

https://www.tensorflow.org/tfx/serving/serving_basic?hl=en

builder.add_meta_graph_and_variables(
    sess, [tf.compat.v1.saved_model.tag_constants.SERVING],
    signature_def_map={
        'predict_images':
            prediction_signature,
        tf.compat.v1.saved_model.signature_constants
            .DEFAULT_SERVING_SIGNATURE_DEF_KEY:
            classification_signature,
    },