The Generative Adversarial Network (GAN) is composed of a generator $G$ and a discriminator $D$. How do these two components interact? What is the intuition behind the GAN, its purpose, and how it is trained?

  • $\begingroup$ Here is a question asking about how GANs and, in particular, the discriminator are trained. $\endgroup$ – nbro Nov 22 '20 at 12:35

GANs were invented in a bar somewhere in Montreal, Canada. At said bar, the idea was that neural networks could be used for generating new examples from an existing distribution. This was the problem:

Given an input set $X$, can we make a new $x'$ that looks like it should be in $X$?

The classic description of a GAN is a counterfeiter (generator) and a cop (discriminator). The counterfeiter has the same problem, make a piece of paper look like a real currency.

In training a GAN, the input to the generator is random noise, a starting seed so that no 2 results are the same. The generator then makes a new $x'$. The input to the discriminator alternates between an actual $x$ and an $x'$ that the generator made. The discriminator then takes the $x'$ and decides whether it is part of the set $X$. The discriminator is then trained using its answer to ensure that it can properly tell the difference between bad counterfeits and elements of $X$. When the discriminator makes a decision on an $x'$ that the generator made, the generator is updated as well, in order to increase its ability to make new $x'$s that the discriminator will think are in $X$.

Using this simple framework, these 2 networks work against each other (adversarially) to train each other until the generator is making $x'$s so well that the discriminator can't tell the difference between them and the real thing. At this point, the generator can be used to make new pictures of cats or whatever was the goal in the first place.

The primary usage of $D$ is training $G$ in a (sorta) supervised way but after $G$ is trained, it isn't needed for generation. That's not to say having a good discriminator isn't useful. There are many applications where that's exactly what you'd want, but, if the goal is generation, then probably not. The generator is trained in a (sorta) supervised fashion because there is actually a label associated with the $x'$ it generates (whether the discriminator could detect that it was a counterfeit or not). Unsupervised methods are used when there are no labels and one wants to understand how their data either by clustering or finding a good distance function. Refer to this answer to see the difference between supervised and unsupervised.

  • $\begingroup$ You could actually say that GANs are trained in a self-supervised way, so you may want to edit this answer to add this info. See this post. Moreover, you should probably cite the article, paper, etc., that states that GANs were invented in a bar. Sure, that story may be true (and I think I already heard of it before), but I think you should link to such original source that talks about it. $\endgroup$ – nbro Nov 22 '20 at 12:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.