I was reading the paper called Improved Techniques for Training GANs. And, in the one-sided label smoothing part, they said that optimum discriminator with label smoothing is

$$ D^*(x)=\frac{\alpha \cdot p_{data}(x) + \beta \cdot p_{model}(x)}{p_{data}(x) + p_{model}(x)}$$

I could not understand where this is come from. How did we get this result?

Note: By the way, I knew how to find optimal discriminator in vanilla GAN i.e. $$ D^*(x) = \frac{p_{r}(x)}{p_{r}(x) + p_g(x)} $$


The equation most likely comes from one of the following references:

David Warde-Farley and Ian Goodfellow. Adversarial perturbations of deep neural networks. In Tamir Hazan, George Papandreou, and Daniel Tarlow, editors, Perturbations, Optimization, and Statistics, chapter 11. 2016. Book in preparation for MIT Press.

C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna. Rethinking the Inception Architecture for Computer Vision. ArXiv e-prints, December 2015.

I was not able to retrieve the book but Szegedy et al. discuss on page 6 (see "Model Regularization via Label Smoothing").

Ian Goodfellow discusses this topic on page 31 of his tutorial:

I. Goodfellow. NIPS 2016 Tutorial: Generative Adversarial Networks. ArXiv e-prints April 2017.

A similar question was asked and answered in the Data Science community: https://datascience.stackexchange.com/questions/28764/one-sided-label-smoothing-in-gans

  • $\begingroup$ Thanks for your answer, but I was expecting someone to explain step by step formulation of this result. Still, I got helpful links $\endgroup$ – Enes Dec 30 '20 at 19:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.