How one can quickly verify that a CNN actually learns?

Question

I tried to build a CNN from scratch based on LeNet architecture from this article

I implemented backdrop and now trying to train it on the MNIST dataset using SGD with 16 batch size. I want to find a quick way to verify that the learning goes well and there are no bugs. For this, I visualize loss for every 100th batch but it takes too long on my laptop and I don't see an overall dynamic (the loss fluctuates downwards, but occasionally jumps up back so I am not sure). Could anyone suggest a proven way to find that the CNN works well without waiting many hours of training?

Answer 1

The MNIST consist of 60k datasets of 28 * 28 pixel.Training a CNN with batch size 16 will have 4000 forward pass per epochs. Now taking into consideration that your are using LeNet which not a very deep model. I would suggest you to do followings:

1. Check your PC specifications such as RAM,Processor,GPU etc.
2. Try your to train your model on cloud service such Google Colab, Kaggle and others
3. Try a batch size of 128 or 64
4. Try to normalize your image data set before training

Training speed also depends on machine learning framework you are using such as Tensorflow, Pytorch etc. I hope this will help.