I am new to pytorch and I am working on DQN for a timeseries using Reinforcement Learning and I needed to have a complex observation of timeseries and some sensor readings, so I merged two neural networks and I am not sure if that's what is ruining my loss.backward or something else.
I know there is multiple questions with the same title but none worked for me, maybe I am missing something.
First of all, this is my network:
class DQN(nn.Module):
def __init__(self, list_shape, score_shape, n_actions):
super(DQN, self).__init__()
self.FeatureList = nn.Sequential(
nn.Conv1d(list_shape[1], 32, kernel_size=8, stride=4),
nn.ReLU(),
nn.Conv1d(32, 64, kernel_size=4, stride=2),
nn.ReLU(),
nn.Conv1d(64, 64, kernel_size=3, stride=1),
nn.ReLU(),
nn.Flatten()
)
self.FeatureScore = nn.Sequential(
nn.Linear(score_shape[1], 512),
nn.ReLU(),
nn.Linear(512, 128)
)
t_list_test = torch.zeros(list_shape)
t_score_test = torch.zeros(score_shape)
merge_shape = self.FeatureList(t_list_test).shape[1] + self.FeatureScore(t_score_test).shape[1]
self.FinalNN = nn.Sequential(
nn.Linear(merge_shape, 512),
nn.ReLU(),
nn.Linear(512, 128),
nn.ReLU(),
nn.Linear(128, n_actions),
)
def forward(self, list, score):
listOut = self.FeatureList(list)
scoreOut = self.FeatureScore(score)
MergedTensor = torch.cat((listOut,scoreOut),1)
return self.FinalNN(MergedTensor)
I have a function called calc_loss, and at its end it return the MSE loss as below
print(state_action_values.dtype)
print(expected_state_action_values.dtype)
return nn.MSELoss()(state_action_values, expected_state_action_values)
and the print shows float32 and float64 respectively.
I get the error when I run the loss.backward() as below
LEARNING_RATE = 0.01
optimizer = optim.Adam(net.parameters(), lr=LEARNING_RATE)
for i in range(50):
optimizer.zero_grad()
loss_v = calc_loss(sample(obs, 500, 200, 64), net, tgt_net)
print(loss_v.dtype)
print(loss_v)
loss_v.backward()
optimizer.step()
and the print output is as below:
torch.float64
tensor(1887.4831, dtype=torch.float64, grad_fn=)
Update 1:
I tried using a simpler model, yet the same issue, when I tried to cast the inputs to Float, I got an error:
RuntimeError: expected scalar type Double but found Float
What makes the model expects double ?
Update 2:
I tried to add the below line on top after the torch import but same issue of RuntimeError: Found dtype Double but expected Float
>>> torch.set_default_tensor_type(torch.FloatTensor)
But when I used the DoubleTensor I got:
RuntimeError: Input type (torch.FloatTensor) and weight type (torch.DoubleTensor) should be the same or input should be a MKLDNN tensor and weight is a dense tensor
The issue wasn't in the input to the network but the criterion of the MSELoss, so it worked fine after casting the criterion to float as below
return nn.MSELoss()(state_action_values.float(), expected_state_action_values.float())
I decided to leave the answer for beginners like me who might be stuck and didn't expect to check the datatype of the loss criterion