Search code examples
pythonmachine-learningpytorchtensor

pytorch .stack final shape after .squeeze


I had a pandas dataframe 200 columns by 2500 rows which I made it into a tensor

tensor = torch.tensor(df.values)
tensor.size() => ([2500,200])

which i chunked and enumerated

list=[]
for i,chunk in enumerate(tensor.chunk(100,dim=0))
    chunk.size =>([25,200])
    output = hiddenlayer(chunks)
    output.size() => ([25,1])
    list += output

chunks were fed through some layers and outputted as 1 feature tensors. So now I have a list of 100 tensors, each with 25 blocks of 1, 100x25x1

so i

stacked = torch.stack(list, 1).squeeze(2)
stacked.size()=([25,100])

I've played around with the stacking and squeezing but i can't seem to get back to ([2500,1]) which is what I want. Am I missing something? If you could quickly help me understand what stacking and squeezing is doing and why it's not working for me I'd be forever in your debt! Thanks


Solution

  • Renaming list to tensor_list since it's bad practice to use reserved keywords as variable names.

    tensor_list =[]
    for i,chunk in enumerate(tensor.chunk(100,dim=0)):
        output = hiddenlayer(chunk).squeeze()
        tensor_list.append(output)
    
    result = torch.reshape(torch.stack(tensor_list,0), (-1, 1))
    

    result.size() should now return torch.Size([2500, 1])