I had a pandas dataframe 200 columns by 2500 rows which I made it into a tensor
tensor = torch.tensor(df.values)
tensor.size() => ([2500,200])
which i chunked and enumerated
list=[]
for i,chunk in enumerate(tensor.chunk(100,dim=0))
chunk.size =>([25,200])
output = hiddenlayer(chunks)
output.size() => ([25,1])
list += output
chunks were fed through some layers and outputted as 1 feature tensors. So now I have a list of 100 tensors, each with 25 blocks of 1, 100x25x1
so i
stacked = torch.stack(list, 1).squeeze(2)
stacked.size()=([25,100])
I've played around with the stacking and squeezing but i can't seem to get back to ([2500,1]) which is what I want. Am I missing something? If you could quickly help me understand what stacking and squeezing is doing and why it's not working for me I'd be forever in your debt! Thanks
Renaming list
to tensor_list
since it's bad practice to use reserved keywords as variable names.
tensor_list =[]
for i,chunk in enumerate(tensor.chunk(100,dim=0)):
output = hiddenlayer(chunk).squeeze()
tensor_list.append(output)
result = torch.reshape(torch.stack(tensor_list,0), (-1, 1))
result.size() should now return torch.Size([2500, 1])