Search code examples
neural-networkpytorchmnist

Inconsitencies/Inaccuracies between torchvision.datasets.MNIST and Michael Nielsens neuralnetworksanddeeplearning

I printed the vectorized form of the first training image of the mnist dataset of pytorch and https://github.com/mnielsen/neural-networks-and-deep-learning/tree/master/data . The difference seems too big for just floating point precision error.

Full Diff of first mnist train image: https://www.diffchecker.com/6y6YTFiN

Code to reproduce:

import mnist_loader
import numpy
numpy.set_printoptions(formatter={'float': '{: 0.4f}'.format})

d = mnist_loader.load_data()
print numpy.array(d[0][0][0]).reshape(784,1)

pytorch:

import torch
import torchvision
from torchvision import transforms as tf

dtype = torch.float32

transforms = tf.Compose([tf.ToTensor()])

mnist_data = torchvision.datasets.MNIST("./mnist", transform=transforms, download=True)
data_loader = torch.utils.data.DataLoader(mnist_data,
                                          batch_size=500,
                                          shuffle=False,
                                          num_workers=1)

data, target = next(data_loader.__iter__())
print(data[0].reshape(784,1))

Anyone an idea what's causing this ?

Solution

  • MNIST images consist of pixel values that are integers in the range 0 to 255 (inclusive). To produce the tensors you are looking at, those integer values have been normalised to lie between 0.0 and 1,0, by dividing them all by some constant factor. It appears that your two sources chose different normalising factors: 255 in one case and 256 in the other.