Pytorch Beginner: TypeError in loss function

Question

I'm a beginner in pytorch. I ran into this RuntimeError and I'm struggling to resolve it. It says that the "result type" of the loss function is Float and cannot be cast to Long. I have tried casting from float32 to LongTensor before the loss function runs, which resulted in another error inside the model("RuntimeError: mat1 and mat2 must have the same dtype"). I do not know how to resolve this Problem.

This is my code:

# Setup loss function
loss_fn = nn.BCEWithLogitsLoss()

# Random Seed
torch.manual_seed = RANDOM_SEED
torch.cuda.manual_seed = RANDOM_SEED

# Epochs
epochs = 1000

# Send data to device
X_train, X_test = X_train.to(device), X_test.to(device)
y_train, y_test = y_train.to(device), y_test.to(device)
# X_train, X_test = X_train.long(), X_test.long()
# y_train, y_test = y_train.long(), y_test.long()


# Training and testing loop
for epoch in range(epochs):
  model_0.train()
  
  # 1. Forward pass (logits -> prob -> label)
  y_logits = model_0(X_train).squeeze()
  y_pred = torch.round(torch.sigmoid(y_logits))

  # 2. Calculate loss and accuracy
  loss = loss_fn(y_logits,
                 y_train)
  acc = acc_fn(y_pred,
               y_train.int())

  # 3. Zero gradients
  optimizer.zero_grad()

  # 4. Loss backward (perform backpropagation)
  loss.backward()

  # 5. Optimizer step in gradient descent
  optimizer.step() 

  ### Testing
  model_0.eval()
  with torch.inference_mode():
    # 1. Forward pass
    test_logits = model_0(X_test).squeeze()
    test_pred = torch.round(torch.sigmoid(test_logits))

    # 2. Calculate test loss
    test_loss = loss_fn(test_logits,
                        y_test)
    test_acc = acc_fn(test_pred,
                      y_test)
    
    if epoch % 100 == 0:
      ep = str(epoch).zfill(4)
      print(f"Epoch: {ep} | Loss: {loss:.5f}, Acc: {acc:.2f}% | Test loss: {test_loss:.5f}, Test acc: {test_acc:.2f}%")

This is the error message I get:

RuntimeError                              Traceback (most recent call last)
<ipython-input-19-e09e5cb6e9d7> in <cell line: 13>()
     19 
     20   # 2. Calculate loss and accuracy
---> 21   loss = loss_fn(y_logits,
     22                  y_train)
     23   acc = acc_fn(y_pred,

2 frames
/usr/local/lib/python3.9/dist-packages/torch/nn/functional.py in binary_cross_entropy_with_logits(input, target, weight, size_average, reduce, reduction, pos_weight)
   3163         raise ValueError("Target size ({}) must be the same as input size ({})".format(target.size(), input.size()))
   3164 
-> 3165     return torch.binary_cross_entropy_with_logits(input, target, weight, pos_weight, reduction_enum)
   3166 
   3167 

RuntimeError: result type Float can't be cast to the desired output type Long

Runtime Error Screenshot

Answer 1

Yes, this is very confusing. Actually what it wants are both in floats. You can pass then to float just using samething like this: y_train.float()