Difference in output when for pytorch NN on batch and individual forward pass

Question

I have the following program:

import torch
import numpy as np

import torch
import torch.nn as nn
import torch.nn.functional as F

class Net(nn.Module):
    def __init__(self, input_size, output_size):
        super(Net, self).__init__()
        self.net = nn.Sequential()
        self.fc1 = nn.Linear(input_size, 32)
        self.fc2 = nn.Linear(32, 32)
        self.fc3 = nn.Linear(32, output_size)

    def forward(self, x):
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        return x

net = Net(input_size=10, output_size=1)
x = torch.from_numpy(np.random.rand(2,10)).type(torch.FloatTensor)

y = net(x)
x0 = x[0,]
y0 = net(x0)

print(y[0,],y0)

assert y0.item() == y[0,].item(), "Outputs not equal"

In the program even after giving the same inputs once in batch and once without batch, why does the NN return two different outputs? Although the difference is very small, but I was curious as to why this happens?

I was expecting the output to be exactly the same as all the computations are deterministic!

Answer 1

The key reason for these variations is the finite precision of floating-point arithmetic in computer systems. While neural networks are theoretically deterministic, these numerical approximations can lead to tiny discrepancies.

Pytorch numerical accuracy: https://pytorch.org/docs/stable/notes/numerical_accuracy.html