1. Weighted Loss in CrossEntropyLoss() 2. Combination of WeightedRandomSampler and subsampler

Question

I wanted to implement class weights to my 3 class classification problem.

Tried by just directly adding the weights, which gives me an error when passing my model output and the labels to my loss

criterion = nn.CrossEntropyLoss(weight=torch.tensor([1,2,2]))

The error:

loss = criterion(out, labels)         
expected scalar type Float but found Long

So I print dtypes and change them to float but it still gives me the same error

labels = labels.float()
print("Labels Training", labels, labels.dtype)
print("Out Training ", out, out.dtype)
loss = criterion(out, labels) 

>>Labels Training tensor([2.]) torch.float32
>>Out Training  tensor([[ 0.0540, -0.1439, -0.0070]], grad_fn=<AddmmBackward0>) torch.float32
>>expected scalar type Float but found Long

I also tried to change it to float64(), but it tells me that tensor Object has no attribute float64

2. Problem: I Havent tried this one out but I have seen that the more used approach would be the RandomWeightedSampler. My problem is that I use CV with K-Fold and use a SubSampler for that. Is it possible to use both? Havent foudn anything related to that.

Answer 1

For the first Problem, nn.CrossEntropyLoss requires the output to be of type float, label of type long, and weight of type float. Therefore, you should change the optional parameter of nn.CrossEntropyLoss "weight" to be float by:

criterion = nn.CrossEntropyLoss(weight=torch.tensor([1.0,2.0,2.0]))
loss = criterion(out, labels.long())