How do I log the outputs of hidden Keras layer during training?

I want to store the float values in the outputs of a particular hidden layer during training. However, since the outputs are KerasTensor objects, I am unable to access them.

How do I access the float values in the tensors so I can store them for later use?

I am currently trying to do this using a custom Callback:

class HidInps(Callback):
    def on_train_batch_end(self, batch, logs=None):
        layer_out = self.model.get_layer("hidlyr").output
        print(layer_out)  # KerasTensor(type_spec=TensorSpec(shape=(None, 3), dtype=tf.float32...
        print(keras.backend.get_value(layer_out))

However, since the KerasTensor object provides no .numpy() method, eval() or get_value() can not work and I get the appropriate error:

AttributeError: 'KerasTensor' object has no attribute 'numpy'

Solution

You need to use the custom training loop in tensorflow to acheive this thing. Lets say your model instance is refered by the variable my_model. You can create another custom_model from it as follows:

from tensorflow.keras import Model

hidden_layer = self.model.get_layer("hidlyr")
custom_model = Model(inputs=my_model.inputs, outputs=[hidden_layer.output, my_model.output])
with tf.GradientTape() as t:
    layer_op, predictions = custom_model(images)
print(layer_op)

For further details, refer https://www.tensorflow.org/tutorials/customization/custom_training_walkthrough#train_the_model