Pytorch Where Does resNet add values?

Question

I am working on ResNet and I have found an implementation that does the skip connections with a plus sign. Like the following

Class Net(nn.Module):
    def __init__(self):
        super(Net, self).__int_() 
            self.conv = nn.Conv2d(128,128)

    def forward(self, x):
        out = self.conv(x) // line 1 
        x = out + x    // skip connection  // line 2

Now I have debugged and printed the values before and after line 1. The output was the following:

after line 1
x = [1,128,32,32]
out = [1,128,32,32]

After line 2
x = [1,128,32,32] // still

Reference link: https://github.com/kuangliu/pytorch-cifar/blob/bf78d3b8b358c4be7a25f9f9438c842d837801fd/models/resnet.py#L62

My question is where did it add the value ?? I mean after

x = out + x

operation, where has the value been added ?

PS: Tensor format is [batch, channel, height, width].

Answer 1

As mentioned in comments by @UmangGupta, what you are printing seems to be the shape of your tensors (i.e. the "shape" of a 3x3 matrix is [3, 3]), not their content. In your case, you are dealing with 1x128x32x32 tensors).

Example to hopefully clarify the difference between shape and content :

import torch

out = torch.ones((3, 3))
x = torch.eye(3, 3)
res = out + x

print(out.shape)
# torch.Size([3, 3])
print(out)
# tensor([[ 1.,  1.,  1.],
#         [ 1.,  1.,  1.],
#         [ 1.,  1.,  1.]])
print(x.shape)
# torch.Size([3, 3])
print(x)
# tensor([[ 1.,  0.,  0.],
#         [ 0.,  1.,  0.],
#         [ 0.,  0.,  1.]])
print(res.shape)
# torch.Size([3, 3])
print(res)
# tensor([[ 2.,  1.,  1.],
#         [ 1.,  2.,  1.],
#         [ 1.,  1.,  2.]])