I am new to neural networks and I don't know exactly how to interpret what I'm getting as results for the validation loss. I'm trying to classify images using tensorflow. If I plot the results I'm getting after each epoch I get the following results: enter image description here
My training accuracy and validation accuracy increase and my training loss decreases but the validation loss has some pikes although it goes down but not very close to the training loss.
How should I interpret this? I don't understand the changes in the validation Loss. Does the fact that the validation Loss is not decreasing as much as the training loss means that I am having overfitting?
(just in case, I'm doing 25 epochs, batch size:128, Learning rate:0.0001 and training/validation split: 0.4)
Thanks for the help
Spiking in the validation loss is not that unusual especially in the early epochs. Generally over fitting is denoted by the situation where your training loss decreases but the validation loss plateaus then starts to increase for each epoch. In general the Training accuracy is usually higher than the validation accuracy and training loss is lower than validation loss. You can do a couple of things to help with improving the validation loss. If the model is over fitting add dropout layers or if you have multiple hidden dense layers just use 1 initially then add more if the training accuracy is poor. The less complex the model the less chance of over fitting. Also using an adjustable learning rate helps. The Keras callback ReduceLROnPlateau can be set up to monitor validation loss and reduce the learning rate if the loss fails to decrease. Documentation is here. Use the callback ModelCheckpoint to save the model with the lowest validation loss and use that model to make predictions on the test set. Documentation is here.
