Search code examples
machine-learningmnist

Neural Network guesses no number for MNIST dataset

I have built my own neural network in java.
Its neuron layers follow this structure: 784(inputs), 200, 80, 10(outputs)

After feeding it the MNIST training data in 300 batches of 100 randomly selected images, updating weights and biases after each batch. With a learning rate of .005. However, the network seems to adopt a strategy of giving an output of all zeros every time. Because just saying
{0,0,0,0,0,0,0,0,0,0}
is much closer to
{0,1,0,0,0,0,0,0,0,0}
than any actual guessing strategy that it tried. On occasion, it will attempt to change but can never find a strategy that works better than that of just saying zero for everything.
Can anyone tell me how to fix this? Does it need more training data? Does this mean there's an error in the backpropagation function I wrote?

Thanks for any suggestions!

Solution
    1. Make sure your data label is integer encoded
    2. Make sure the final Dense layer has 10 units with a softmax activation function.
    3. Compile your model with sparse_categorical_crossentropy as the loss function.