Alternatives for loss functions in python CNTK

Question

I have created a sequential model in CNTK and pass this model into a loss function like the following:

ce = cross_entropy_with_softmax(model, labels)

As mentioned here and as I have multilabel classifier, I want to use a proper loss function. The problem is I can not find any proper document to find these loss functions in Python. Is there any suggestion or sample code for this requirement.

I should notice that I found these alternatives (logistic and weighted logistic) in BrainScript language, but not in Python.

Answer 1

"my data has more than one label (three label) and each label has more than two values (30 different values)"

Do I understand right, you have 3 network outputs and associated labels, and each one is a 1-in-30 classifier? Then it seems you can just add three cross_entropy_with_softmax() values. Is that what you want?

E.g. if the model function returns a triple (ending in something like return combine([z1, z2, z3])), then your criterion function that you pass to Trainer could look like this (if you don't use Python 3, the syntax is a little different):

from cntk.layers.typing import Tensor, SparseTensor
@Function
def my_criterion(input : Tensor[input_dim], labels1 : SparseTensor[30],
                 labels2 : SparseTensor[30], labels3 : SparseTensor[30]):
    z1, z2, z3 = my_model(input).outputs
    loss = cross_entropy_with_softmax(z1, labels1) + \
           cross_entropy_with_softmax(z2, labels2) + \
           cross_entropy_with_softmax(z3, labels3)
    return loss

learner = ...
trainer = Trainer(None, my_criterion, learner)

# in MB loop:
input_mb, L1_mb, L2_mb, L3_mb = my_next_minibatch()
trainer.train_minibatch(my_criterion.argument_map(input_mb, L1_mb, L2_mb, L3_mb))