Search code examples
pythontensorflowkeraslossregularized

Is there a way to add keras 'custom layer' based/specific penalty to the overall loss function?

i have a keras sequential model with some custom layers in it. Now in one of the layers, based on the input of that specific layer, i want to calculate a penalty and i want the penalty to be added to the loss function which the optimizer tries to minimize overall.

I have gone through the concept of tf.keras.layers.ActivityRegularization but struggling to figure out how to solve my issue.

Solution

  • If you want to "add", you just need to calculate the layer output/loss, and use model.add_loss(loss_tensor) 

    ....
loss_tensor = MyCustomLayer(...)(layer_inputs)
....

model = Model(model_inputs, model_outputs)
model.add_loss(loss_tensor)
model.compile(loss=any_normal_loss)