Search code examples
neural-networkkerasregressionmse

Training keras model; why mae decreases while mse increases?

I am training a model using keras on a regression problem. When I investigate the loss and metrics during training, sometimes mean absolute error (mae) decreases at the end of an epoch, while mean square error (mse) increases. I set mae as loss and mse as metric.

Is it OK? Or is there any problem with the setting? Thanks

Solution

  • MSE and MAE are different metrics. A decrease in the one does not imply a decrease in the other. Consider the following toy example for the size-2 output values of a network with the target value as Target: [0,0]

    • Timestep 1: Output: [2,2], MAE: 2, MSE: 4
    • Timestep 2: Output: [0,3], MAE: 1.5, MSE: 4.5

    So MAE decreased while MSE increased. Given that you are optimizing for MAE and only monitor MSE, your observation is perfectly fine and does not imply any problem.