How to pass the input tensor of a model to a loss function?

Question

My goal is to create a custom loss function that calculates the loss based on y_true, y_pred and the tensor of the models input layer:

import numpy as np
from tensorflow import keras as K

input_shape = (16, 16, 1)

input = K.layers.Input(input_shape)
dense = K.layers.Dense(16)(input)
output = K.layers.Dense(1)(dense)

model = K.Model(inputs=input, outputs=output)


def CustomLoss(y_true, y_pred):
  return K.backend.sum(K.backend.abs(y_true - model.input * y_pred))


model.compile(loss=CustomLoss)
model.fit(np.ones(input_shape), np.zeros(input_shape))

However, this code fails with the following error message:

TypeError: Cannot convert a symbolic Keras input/output to a numpy array. This error may indicate that you're trying to pass a symbolic value to a NumPy call, which is not supported. Or, you may be trying to pass Keras symbolic inputs/outputs to a TF API that does not register dispatching, preventing Keras from automatically converting the API call to a lambda layer in the Functional Model.

How can I pass the input tensor of my model to the loss function?

Tensorflow Version: 2.4.1
Python Version: 3.8.8

Answer 1

You can use add_loss to pass external layers to your loss. Here an example:

import numpy as np
from tensorflow import keras as K

def CustomLoss(y_true, y_pred, input_l):
    return K.backend.sum(K.backend.abs(y_true - input_l * y_pred))

input_shape = (16, 16, 1)
n_sample = 10

X = np.random.uniform(0,1, (n_sample,) + input_shape)
y = np.random.uniform(0,1, (n_sample,) + input_shape)

inp = K.layers.Input(input_shape)
dense = K.layers.Dense(16)(inp)
out = K.layers.Dense(1)(dense)

target = K.layers.Input(input_shape)
model = K.Model(inputs=[inp,target], outputs=out)

model.add_loss( CustomLoss( target, out, inp ) )
model.compile(loss=None, optimizer='adam')
model.fit(x=[X,y], y=None, epochs=3)

To use the model in inference mode (removing the target from inputs)

final_model = K.Model(model.input[0], model.output)
final_model.predict(X)