Can intermediate layers be accessed directly within keras loss function?

Question

I am curious whether a loss function can implement intermediate layer outputs within keras, without designing the model to feed the intermediate layers as outputs. I have seen a solution can be to redesign the architecture to return the intermediate layer in addition to the final prediction and use that as a workaround, but I'm unclear whether a layer output can be accessed directly from a loss function

Answer 1

I'm unclear whether a layer output can be accessed directly from a loss function

It certainly can.

By way of an example, consider this model using the functional API:

inp = keras.layers.Input(shape=(28, 28))
flat = keras.layers.Flatten()(inp)
dense = keras.layers.Dense(128, activation=tf.nn.relu)(flat)
out = keras.layers.Dense(10, activation=tf.nn.softmax)(dense)

model = keras.models.Model(inputs=inp, outputs=out )
model.compile(optimizer='adam', 
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])

If, say, we wanted to introduce a new loss function that also penalised the largest weight of the outputs of our dense layer then we could write a custom loss function something like this:

def my_funky_loss_fn(y_true, y_pred):
  return (keras.losses.sparse_categorical_crossentropy(y_true, y_pred) 
        + keras.backend.max(dense))

which we can use in our model just by passing our new loss function to the compile() method:

model.compile(optimizer='adam', 
              loss=my_funky_loss_fn,
              metrics=['accuracy'])