How visualize in Tensorboard a metric callback?

Question

I have a model in keras in which I use my custom metric as:

class MyMetrics(keras.callbacks.Callback):
    def __init__(self):
        initial_value = 0
    def on_train_begin(self, logs={}):
        ...
    def on_epoch_end(self, batch, logs={}):
        here I calculate my important values

Now, there is a way to visualize them in Tensorboard? For example if my metric was something like:

def mymetric(y_true,y_pred):
    return myImportantValues

I could visualize them in Tensorboard through mymodel.compile(..., metrics = mymetric)

Is there something similar with a metric callback? I tried to create a function inside the class MyMetric and pass it to the mymodel.compile but it does not update the values.

Answer 1

You can create an event file with the custom metrics and visualize it in tensorboard directly.

This works for Tensorflow 2.0. In this example, the accuracy/metrics are logged from training history. In your case, you can do it from the on_epoch_end callback.

import datetime
current_time = datetime.datetime.now().strftime("%Y%m%d-%H%M%S")
train_log_dir = 'logs/train/' + current_time
train_summary_writer = tf.summary.create_file_writer(train_log_dir)

history = model.fit(x=X, y=y, epochs=100, verbose=1)
for epoch in range(len(history.history['accuracy'])):
    with train_summary_writer.as_default():
        tf.summary.scalar('loss', history.history['loss'][epoch], step=epoch)
        tf.summary.scalar('accuracy', history.history['accuracy'][epoch], step=epoch)

After script execution,

tensorboard --logdir logs/train

https://www.tensorflow.org/tensorboard/r2/get_started#using_tensorboard_with_other_methods