Keras custom loss function: ValueError in a tf.function-decorated

Question

I'm trying to implement in TF 2.2 the loss function from this paper (an existing version in TensorFlow 1.10.1, made by the author of the paper can be found here).

However, the theoretical details of the loss function are not relevant to my problem.

My loss function:

def z_score_based_deviation_loss(y_true, y_pred):
    confidence_margin = 5.0     
    ref = K.variable(np.random.normal(loc=0., scale=1.0, size=5000), dtype='float32')
    dev = (y_pred - K.mean(ref)) / K.std(ref)
    inlier_loss = K.abs(dev) 
    outlier_loss = K.abs(K.maximum(confidence_margin - dev, 0.))
    return K.mean((1 - y_true) * inlier_loss + y_true * outlier_loss)

Calling it by:

model.compile(optimizer='adam', loss=z_score_based_deviation_loss)

I got this error:

ValueError: tf.function-decorated function tried to create variables on non-first call.

I know there are other questions on this subject, but I haven't found any that are related to a custom loss function in Keras. I can't figure out how to adapt it.

Also, from what I've read, the problem should exists when the function is tagged with @tf.function (which is recommended anyway to speed up the computation), but I didn't add it...

Answer 1

try in this way

def z_score_based_deviation_loss(y_true, y_pred):
    confidence_margin = 5.0     
    ref = tf.random.normal([5000], mean=0., stddev=1.)
    dev = (y_pred - K.mean(ref)) / K.std(ref)
    inlier_loss = K.abs(dev) 
    outlier_loss = K.abs(K.maximum(confidence_margin - dev, 0.))
    return K.mean((1 - y_true) * inlier_loss + y_true * outlier_loss)


X = np.random.uniform(0,1, (100,10))
y = np.random.uniform(0,1, 100)

x_input = Input((10,))
intermediate = Dense(1, activation='linear',  name = 'score')(x_input)
model = Model(x_input, intermediate)

model.compile('adam', z_score_based_deviation_loss)
model.fit(X, y, epochs=10)