Search code examples
pythontensorflowkerasloss-function

Custom loss function produces extremely low loss value with Keras 2.1.4

I'm working with a dataset of numerical and categorical values (time series). This is a sample of the variables:
A B C_1 C_2 D_1 D_2 D_3
First two are numerical variables, C and D are categorical variables with a onehot representation.

Below my custom loss function. I use partial in order to pass more than two parameters to the function:

def mixed_num_cat_loss_backend(y_true, y_pred, signals_splits):
    if isinstance(y_true, np.ndarray):
        y_true = keras.backend.variable( y_true )
    if isinstance(y_pred, np.ndarray):
        y_pred = keras.backend.variable( y_pred )

    y_true_mse = y_true[:,:signals_splits[0]] 
    y_pred_mse = y_pred[:,:signals_splits[0]]
    mse_loss_v = keras.backend.square(y_true_mse-y_pred_mse)

    categ_loss_v = [ keras.backend.categorical_crossentropy(
                             y_pred[:,signals_splits[i-1]:signals_splits[i]], #keras.backend.softmax(y_pred[:,signals_splits[i-1]:signals_splits[i]]), 
                             y_true[:,signals_splits[i-1]:signals_splits[i]], 
                         from_logits=False) # force keras to normalize
                    for i in range(1,len(signals_splits)) ]

    losses_v = keras.backend.concatenate( [mse_loss_v, keras.backend.stack(categ_loss_v,1)], 1)

    return losses_v

After one epoch I have an extremely low loss value:

Epoch 1/100
76s - loss: 0.1040 - acc: 0.1781 - val_loss: 0.0016 - val_acc: 0.1330  
Epoch 2/100  
75s - loss: 9.2523e-04 - acc: 0.1788 - val_loss: 8.7442e-04 - val_acc: 0.1330

The point is that I don't have this problem when I use Keras 2.0.4.

Solution

  • The signature of cross-entropy backend methods has changed since Keras 2.0.7. According to the release note,

    The backend methods categorical_crossentropy, sparse_categorical_crossentropy, binary_crossentropy had the order of their positional arguments (y_true, y_pred) inverted. This change does not affect the losses API. This change was done to achieve API consistency between the losses API and the backend API.

    So you should switch the place of y_true and y_pred when calling categorical_crossentropy in newer versions of Keras.