Im am trying to learn and also adapt the ImageNeuralNetwork example in Java. So far my problem is that when i give the NN a larger amount of images that are 32X32 and let it train the error never goes down below 14% and at the beginning it jumps all over the place.
My images are BW and they are classified into 27 classes. so i know that there are 27 output neurons.
My question is why is the NN not learning, I tried setting different hidden layers ( 1 or 2 layers) with different neuron counts but nothing helps.
Can anyone give me an idea of what im doing wrong? Like i said im just beginning with NNs and im a bit lost here
Edit: It seems if i give it less images as input to learn the error goes down, but that doesnt really solve the problem, If i wanted to classify a lot of images i would be stuck with the error never going down
You need to use only one hidden layer. Additional hidden layers on a neural network really does not give you much, see the universal approximation theorem. I would try starting with (input count + output count) * 1.5 as the number of hidden neurons.
As to why the ANN fails to coverage with more images, that is more difficult. Most likely it is because the additional images are too varied for the ANN to classify them all. A simple feedforward ANN is really not ideal for grid-based image recognition. The neural network does not know which pixels are next to each other, it is just a straight linear vector of pixels. The ANN is basically learning which pixels must be present for each of the letters. If you shift one of the letters even slightly, the ANN might not recognize it because you've now moved nearly every pixel that it was trained with.
I really do not do much with OCR. However, this does seem to be the area where deep learning excels. Convolutional neural networks are better able to handle pixels near each other and approximate. You might get better results with a deep learning application. More info here: http://dpkingma.com/sgvb_mnist_demo/demo.html