Search code examples
pythonneural-networklasagne

Lasagne - Why is the Validation Loss Highlighted Green?

I'm running the MNIST Neural Network example and when training the classifier, the Validation Loss column (and training loss) has some values highlighted in green. If i have the learning rate at 0.01, all are green, however if i increase it to 0.1, only half of them are highlighted.

What does this highlighting mean if anything?

Output example:

Validation Loss Highlighting

Solution

  • Without knowing the internals, i see a pattern there.

    • Color each row as long as there is an improvement (minimization -> it's lower)
      • train-loss blue
      • val-loss green
    • Stop coloring when loss / validation did increase
      • Don't start coloring again after this one-time increase

    Different learning-rates will effect in different train-loss / val-loss paths and therefore some possible different coloring.

    Your example fits well there, as a lower learning-rate is usually more stable (from iteration to iteration) in regards to lowering the loss (more blue, more green).