I'm trying to understand the R1 regularization function, both the abstract concept and every symbol in the formula. According to the article, the definition of R1 is:

It penalizes the discriminator from deviating from the Nash Equilibrium via penalizing the gradient on real data alone: when the generator distribution produces the true data distribution and the discriminator is equal to 0 on the data manifold, the gradient penalty ensures that the discriminator cannot create a non-zero gradient orthogonal to the data manifold without suffering a loss in the GAN game.

$R_1(\psi ) = \frac{\gamma}{2}E_{pD(x)}\left [ \left \| \bigtriangledown D_{\psi}(x) \right \|^2 \right ]$

I have basic understanding of how GAN's and back-propagation works. I understand the idea of punishing the discriminator when he deviates from the Nash equilibrium. The rest of it gets murky, even if it might be basic math. For example, I'm not sure why it matters if the gradient is orthogonal to the data.

On the equation part, it's even more unclear. The discriminator input is always an image, so I assume $x$ is an image. Then what is $\psi$ and $\gamma$?

(I understand this is somewhat of a basic question, but seems there are no blogs about it for us simple non-researchers, math challenged people who fail to understand the original article )

  • $\begingroup$ Rather than providing a link to an external resource, can you just type in latex (yes, you can use latex on this site) of the specific formula that you're referring to? Moreover, please, provide some context, and do not just say "explain X". $\endgroup$ – nbro Dec 30 '20 at 10:43
  • $\begingroup$ @nbro I added latex. I initially avoided adding context because I wanted to keep this question more generic as this really seems to be mostly uncovered by popular blogs. I tried to add some directions but I'm afraid I don't have any more concrete context other than I encountered that StyleGAN uses for one dataset and not for another, and I want to "get a feel" as to how it works. $\endgroup$ – Aviad Hadad Dec 30 '20 at 11:31
  • $\begingroup$ Thank you! Even if you're looking for a general answer, providing some context is always a good idea. $\endgroup$ – nbro Dec 30 '20 at 11:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.