Search code examples
neural-networkpytorchlstmloss-functionsoftmax

Testing and Confidence score of Network trained with nn.CrossEntropyLoss()

I have trained a network with the following structure:

Intent_LSTM(
(attention): Attention()
(embedding): Embedding(34601, 400)
(lstm): LSTM(400, 512, num_layers=2, batch_first=True, dropout=0.5)
(dropout): Dropout(p=0.5, inplace=False)
(fc): Linear(in_features=512, out_features=3, bias=True)
)

Now I want to test this trained network and also get the confidence score of classification. Here is my current implementation of test function:

output = model_current(inputs)
pred = torch.round(output.squeeze())
pred = pred.argmax(dim=1, keepdim=True)

Now my question isas follows.

  1. Here pred is just the output from a Fully connected layer from my network without softmax (as required by a loss function). Is this(pred = pred.argmax(dim=1, keepdim=True)) the right way to get the predictions? Or should I pass the output from the network to a softmax layer and then do argmax?

  2. How do I get the confidence score? Should I pass the output from the network to a softmax layer and select the argmax as the confidence of the class?

Solution
    1. It doesn't really matter if you pick argmax before or after doing softmax. Because whatever maximizes softmax will also maximize the logits (pre-softmax) values. So you should get similar values.
    2. Softmax will give you scores or probabilities for each class. Hence, the values after doing softmax can be used as confidence scores.