I was trying to evaluate my classification models with log-loss metric using mxnet.metric module.
I came across two classes: CrossEntropy and NegativeLogLikelihood which have the same definition and very similar implementation. Both have the same core code for calculating metric value:
self.sum_metric += (-numpy.log(prob + self.eps)).sum()
If they are essentially the same metrics what is the purpose of maintaining both functions in the library? Which metric is preferred for binary and multi-class classification?
There is actually not much difference between these things not only in MXNet, but overall. Read this Quora for more intuition: https://www.quora.com/What-are-the-differences-between-maximum-likelihood-and-cross-entropy-as-a-loss-function
As for what is used for multi-class classification in MXNet, then in majority of examples I have seen people use SoftmaxCrossEntropyLoss. See an example here: https://gluon.mxnet.io/chapter02_supervised-learning/softmax-regression-gluon.html