I have a trained network that I'm using to predict image data. I would like to speed up the prediction, but I don't know how to run the tensorboard profiler on predict. I prefer not to rerun training to optimize because of model size. This is similar to a previous question, but I'm asking again because that question got no relevant answers.
When I follow this profiling example, I get a profile, but when I add a predict step with the same callback it doesn't profile the predict.
(previous lines are loading and setting up an MNIST model as in the linked colab)
tboard_callback = tf.keras.callbacks.TensorBoard(log_dir = logs,
histogram_freq = 1,
profile_batch='500,520')
model.fit(ds_train,
epochs=2,
validation_data=ds_test,
callbacks = [tboard_callback]
)
model.predict(ds_test,callbacks=[tboard_callback])
Environment: Windows 10, python 3.8, tensorflow 2.2, tensorboard_plugin_profile 2.4.0
I can suggest a workaround. Write a custom prediction loop and instrument it with a tf.profiler.experimental.Trace like described in Profiling API section of the Optimize TensorFlow performance using the Profiler guide.
