1
$\begingroup$

I was reading the paper called Improved Techniques for Training GANs. And, in the one-sided label smoothing part, they said that optimum discriminator with label smoothing is

$$ D^*(x)=\frac{\alpha \cdot p_{data}(x) + \beta \cdot p_{model}(x)}{p_{data}(x) + p_{model}(x)}$$

I could not understand where this is come from. How did we get this result?

Note: By the way, I knew how to find optimal discriminator in vanilla GAN i.e. $$ D^*(x) = \frac{p_{r}(x)}{p_{r}(x) + p_g(x)} $$

$\endgroup$
1
  • $\begingroup$ I did not yet write it down in detail but to my understanding deriving this result should work like deriving the optimal discriminator in the vanilla GAN (which does not use label smoothing). In case of label smoothing your integrands just get constants alpha and beta... $\endgroup$
    – Ggjj11
    Jul 9 '21 at 22:01
1
$\begingroup$

The equation most likely comes from one of the following references:

David Warde-Farley and Ian Goodfellow. Adversarial perturbations of deep neural networks. In Tamir Hazan, George Papandreou, and Daniel Tarlow, editors, Perturbations, Optimization, and Statistics, chapter 11. 2016. Book in preparation for MIT Press.

C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna. Rethinking the Inception Architecture for Computer Vision. ArXiv e-prints, December 2015.

I was not able to retrieve the book but Szegedy et al. discuss on page 6 (see "Model Regularization via Label Smoothing").

Ian Goodfellow discusses this topic on page 31 of his tutorial:

I. Goodfellow. NIPS 2016 Tutorial: Generative Adversarial Networks. ArXiv e-prints April 2017.

A similar question was asked and answered in the Data Science community: https://datascience.stackexchange.com/questions/28764/one-sided-label-smoothing-in-gans

$\endgroup$
1
  • $\begingroup$ Thanks for your answer, but I was expecting someone to explain step by step formulation of this result. Still, I got helpful links $\endgroup$
    – Enes
    Dec 30 '20 at 19:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.