Normalize MNIST in PyTorch

Question

I am trying to normalize MNIST dataset in PyTorch 1.9 and Python 3.8 to be between the range [0, 1] with the code (batch_size = 32).

# Specify path to MNIST dataset-
path_to_data = "path_to_dataset"

# Define transformation(s) to be applied to dataset-
transforms_MNIST = transforms.Compose(
    [
        transforms.ToTensor(),
        transforms.Normalize(mean = (0.1307,), std = (0.3081,))
    ]
)

# Load MNIST dataset-
train_dataset = torchvision.datasets.MNIST(
        # root = './data', train = True,
        root = path_to_data + "data", train = True,
        transform = transforms_MNIST, download = True
        )

test_dataset = torchvision.datasets.MNIST(
        # root = './data', train = False,
        root = path_to_data + "data", train = False,
        transform = transforms_MNIST
        )

# Create training and testing dataloaders-
train_loader = torch.utils.data.DataLoader(
        dataset = train_dataset, batch_size = batch_size,
        shuffle = True
        )

test_loader = torch.utils.data.DataLoader(
        dataset = test_dataset, batch_size = batch_size,
        shuffle = False
        )

print(f"Sizes of train_dataset: {len(train_dataset)} and test_dataet: {len(test_dataset)}")
print(f"Sizes of train_loader: {len(train_loader)} and test_loader: {len(test_loader)}")
# Sizes of train_dataset: 60000 and test_dataet: 10000
# Sizes of train_loader: 1875 and test_loader: 313

# Sanity check-
print(f"train_dataset: min pixel value = {train_dataset.data.min().numpy():.3f} &"
      f" max pixel value = {train_dataset.data.max().numpy():.3f}")
# train_dataset: min pixel value = 0.000 & max pixel value = 255.000

print(f"test_dataset: min pixel value = {test_dataset.data.min().numpy():.3f} &"
      f" max pixel value = {test_dataset.data.max().numpy():.3f}")
# test_dataset: min pixel value = 0.000 & max pixel value = 255.000

print(f"len(train_loader) = {len(train_loader)} & len(test_loader) = {len(test_loader)}")
# len(train_loader) = 1875 & len(test_loader) = 313

# Sanity check-
len(train_dataset) / batch_size, len(test_dataset) / batch_size
# (1875.0, 312.5)

# Get some random batch of training images & labels-
images, labels = next(iter(train_loader))

# You get x images due to the specified batch size-
print(f"images.shape: {images.shape} & labels.shape: {labels.shape}")
# images.shape: torch.Size([32, 1, 28, 28]) & labels.shape: torch.Size([32])

# Get min and max values for normalized pixels in mini-batch-
images.min(), images.max()
# (tensor(-0.4242), tensor(2.8215))

The min and max for 'images' should be between 0 and 1, instead, it is 0.4242 and 2.8215. What is going wrong?

Answer 1

This happens because Normalize applies what is actually known (also) as a standardization: output = (input - mean) / std.

The normalization you want to achieve is automatically performed when loading the image so you can comment Normalize.