How to do Inference and Transfer Learning with TensorFlow Frozen GraphDef (single saved_model.pb) from Google AutoML Vision Classification

Question

I am using an exported classification model from Google AutoML Vision, hence I only have a saved_model.pb and no variables, checkpoints etc. I want to load this model graph into a local TensorFlow installation, use it for inference and continue training with more pictures.

Main questions:

• Is this plan possible, i.e. to use a single saved_model.pb without variables, checkpoints etc. and train the resulting graph with new data?

• If yes: How do you get to an input shape of (?,) with images encoded as strings?

• Ideally, looking ahead: Any important thing to consider for the training part?

---

Background infos about code:

• To read the image, I use the same approach as you would when using the Docker container for inference, hence base64 encoded image.

• To load the graph, I checked what tag set the graph needs via CLI (saved_model_cli show --dir input/model) which is serve.

• To get input tensor names I use graph.get_operations(), which gives me Placeholder:0 for image_bytes and Placeholder:1_0 for the key (just an arbitrary string identify the image). Both have Dimension dim -1

import tensorflow as tf
import numpy as np
import base64

path_img = "input/testimage.jpg"
path_mdl = "input/model"

# input to network expected to be base64 encoded image
with io.open(path_img, 'rb') as image_file:
    encoded_image = base64.b64encode(image_file.read()).decode('utf-8')

# reshaping to (1,) as the expecte dimension is (?,)
feed_dict_option1 = {
    "Placeholder:0": { np.array(str(encoded_image)).reshape(1,) }, 
    "Placeholder_1:0" : "image_key"
}

# reshaping to (1,1) as the expecte dimension is (?,)
feed_dict_option2 = {
    "Placeholder:0": np.array(str(encoded_image)).reshape(1,1), 
    "Placeholder_1:0" : "image_key"
}

with tf.Session(graph=tf.Graph()) as sess:
    tf.saved_model.loader.load(sess, ["serve"], path_mdl)

    graph = tf.get_default_graph()

    sess.run('scores:0',
               feed_dict=feed_dict_option1)

    sess.run('scores:0',
               feed_dict=feed_dict_option2)

Output:

# for input reshaped to (1,)
ValueError: Cannot feed value of shape (1,) for Tensor 'Placeholder:0', which has shape '(?,)'

# for input reshaped to (1,1)
ValueError: Cannot feed value of shape (1, 1) for Tensor 'Placeholder:0', which has shape '(?,)'

How do you get to an input shape of (?,)?

Thanks a lot.

Answer 1

Yes! It is possible, I have an object detection model that should be similar, I can run it as follows in tensorflow 1.14.0:

import cv2
cv2.imread(filepath)
flag, bts = cv.imencode('.jpg', img)
inp = [bts[:,0].tobytes()]
out = sess.run([sess.graph.get_tensor_by_name('num_detections:0'),
                sess.graph.get_tensor_by_name('detection_scores:0'),
                sess.graph.get_tensor_by_name('detection_boxes:0'),
                sess.graph.get_tensor_by_name('detection_classes:0')],
               feed_dict={'encoded_image_string_tensor:0': inp})

I used netron to find my input.

In tensorflow 2.0 it is even easier:

import cv2
cv2.imread(filepath)
flag, bts = cv.imencode('.jpg', img)
inp = [bts[:,0].tobytes()]
saved_model_dir = '.'
loaded = tf.saved_model.load(export_dir=saved_model_dir)
infer = loaded.signatures["serving_default"]
out = infer(key=tf.constant('something_unique'), image_bytes=tf.constant(inp))

Also saved_model.pb is not a frozen_inference_graph.pb, see: What is difference frozen_inference_graph.pb and saved_model.pb?