Search code examples
pythonkerasloss-function

Building a custom loss in Keras

I have been trying to build my own custom loss in Keras and encounter some error.

My need is the following: I want to build a classifier on a time series that is stable: it must decide wether the curve has an ascendant or descendant tendency, meaning that if the derivative is negative for say 2 or 3 time steps, it must keep the "ascendant" classification rather than switching to descendant for a small amount of time. The input data are contextual data that should help the NN detect wether the current change in the direction of the time series is likely to last or not.

To do this I imagined building a custom loss function that would add a penalty for each change of classification above or below the reference.

I first wanted to use np.where on the predicted and reference tensors but I understood browsing the forum this was not possible as the loss function uses placeholder tensors. I must remain within the functional space of keras backend.

For this reason, I used some code I found on this forum to build the Following loss function:

import keras.backend as kb

def roll_reg(x):
    length = kb.int_shape(x)[0]
    x_tile = kb.tile(x, [2, 1])
    x_roll = x_tile[length - 1:-1]
    return kb.sum(kb.abs(x - x_roll))

def custom_loss(y_actual,y_predicted):
    posact=kb.flatten(y_actual)
    pospred=kb.flatten(y_predicted)
    na=roll_reg(posact)
    np=roll_reg(pospred)
    loss_cust=kb.mean(((y_actual-y_predicted)**2)**0.5/(kb.max(y_actual)-kb.min(y_actual)))+abs(na-np)/kb.int_shape(posact)*10
    return loss_cust

but I still get this error:

ValueError: Shape must be rank 1 but is rank 2 for 'loss_9/dense_25_loss/Tile' (op: 'Tile') with input shapes: [?], [2].

which I interpret as my operations not being possible on placeholder tensors… while I was trying to use a dedicated function for these kind of tensors.

Does anyone have any idea on what I could do to improve this code in order to get my loss function running?

Some update on this question. I found on the internet this page : https://medium.com/@j.ali.hab/on-custom-loss-functions-in-keras-3af88cf59e48 where the same issue is solved using tf.math Library.

So I updated my code to :

def custom_loss(y_actual,y_predicted):
    na=kb.tf.math.cumsum(y_actual,axis=0)
    np=kb.tf.math.cumsum(y_predicted,axis=0)

    penalty=kb.abs(kb.sum(np-na))/118319#/kb.int_shape(y_actual)[0]
    loss_nico=penalty
    return loss_nico

Now the code is compiling but I get a Nan at the very end of each epoch as a loss function. Of course, the neural net is not learning anything as a consequence.

Does anyone have a clue about what I am doing wrong ?

Solution

  • OK, I finally got it !

    the problem was in te kb.abs() : it is not needed as kb.sum() returns a scalar.

    So the final code that works is :

    def custom_loss(y_actual,y_predicted):

    na=kb.tf.math.cumsum(y_actual,axis=0)
    np=kb.tf.math.cumsum(y_predicted,axis=0)
    penalty=abs(kb.sum(na-np))
    loss_nico=kb.sum(((y_actual-y_predicted)))/1000+penalty/1000
    return loss_nico

    this compiles and enables the RN to actually learn Something. It is still not perfect as the "non penalty" part should be improved but it works still.