Search code examples
deep-learningloss-function

Loss implementation in Improved-GAN

Recently I am reading the paper Improved Techniques for Training GANs, the author define the loss as follows: enter image description here Then I check the code of the article, the corresponding code of loss definition is:

output_before_softmax_lab = ll.get_output(disc_layers[-1], x_lab, deterministic=False)
output_before_softmax_unl = ll.get_output(disc_layers[-1], x_unl, deterministic=False)
output_before_softmax_gen = ll.get_output(disc_layers[-1], gen_dat, deterministic=False)

l_lab = output_before_softmax_lab[T.arange(args.batch_size),labels]
l_unl = nn.log_sum_exp(output_before_softmax_unl)
l_gen = nn.log_sum_exp(output_before_softmax_gen)
loss_lab = -T.mean(l_lab) + 
T.mean(T.mean(nn.log_sum_exp(output_before_softmax_lab)))
loss_unl = -0.5*T.mean(l_unl) + 0.5*T.mean(T.nnet.softplus(l_unl)) + 0.5*T.mean(T.nnet.softplus(l_gen))

I recognize that the l_lab is for the classification loss on labeled data,so it should be minimized, and the l_unl is the loss with regard to unlabeled data, and the l_gen loss on generated images. My confusion is why the discriminator should minimize the l_unl and l_gen,which is the code 0.5*T.mean(T.nnet.softplus(l_unl)) + 0.5*T.mean(T.nnet.softplus(l_gen)) tells us. Thanks in advance.

Solution

  • You should read this fragment of Improve-gan. enter image description here

    Loss_unlabel correspond to L_{unsupervised}.

    See in the below. enter image description here

    Then, by the code of the function nn.softplus and nn.logsum_exp, you will get the code of loss_unl

    Hope to help you.