what is wrong with the following implementation of Conv1d?

Question

I am trying to implement a Conv1d layer with Batch Normalization but I keep getting the following error:

RuntimeError                              Traceback (most recent call last)
<ipython-input-32-ef6e122ea50c> in <module>()
----> 1 test()
      2 for epoch in range(1, n_epochs + 1):
      3   train(epoch)
      4   test()

7 frames
/usr/local/lib/python3.7/dist-packages/torch/nn/modules/conv.py in _conv_forward(self, input, weight, bias)
    258                             _single(0), self.dilation, self.groups)
    259         return F.conv1d(input, weight, bias, self.stride,
--> 260                         self.padding, self.dilation, self.groups)
    261 
    262     def forward(self, input: Tensor) -> Tensor:

RuntimeError: Expected 3-dimensional input for 3-dimensional weight [25, 40, 5], but got 2-dimensional input of size [32, 40] instead

The data is passed on in batches of 32 using DataLoader class and it has 40 features and 10 labels. Here is my model:

class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()
        #self.flatten=nn.Flatten()
        self.net_stack=nn.Sequential(
            nn.Conv1d(in_channels=40, out_channels=25, kernel_size=5, stride=2), #applying batch norm
            nn.ReLU(),
            nn.BatchNorm1d(25, affine=True),
            nn.Conv1d(in_channels=25, out_channels=20, kernel_size=5, stride=2), #applying batch norm
            nn.ReLU(),
            nn.BatchNorm1d(20, affine=True),
            nn.Linear(20, 10),
            nn.Softmax(dim=1))

    def forward(self,x):
        #x=torch.reshape(x, (1,-1))
        result=self.net_stack(x)
        return result

I have tried given in other answers like unsqueezing the input tensor, but none of the models in such questions is using Conv1d with batchnorm1d so I am not able to narrow down the problem to which layer must be causing the error. I have just started with using Pytorch and was able to implement a simple linear NN model, but I am facing this error while using a convolutional NN for the same data.

Answer 1

You need to add a batch dimension to your input (and also change the number of input channels).

A conv1d layer accepts inputs of shape [B, C, L], where B is the batch size, C is the number of channels and L is the width/length of your input. Also, your conv1d layer expects 40 input channels:

nn.Conv1d(in_channels=40, out_channels=25, kernel_size=5, stride=2)

hence, your input tensor x must have shape [B, 40, L] while now it has shape [32, 40].

Try:

def forward(self,x):
    result=self.net_stack(x[None])
    return result

you will get another error complaining about dimensions mismatch, suggesting you need to change the number of input channels to 40.