How are generative adversarial networks trained?

Question

I am reading about generative adversarial networks (GANs) and I have some doubts regarding it. So far, I understand that in a GAN there are two different types of neural networks: one is generative ($G$) and the other discriminative ($D$). The generative neural network generates some data which the discriminative neural network judges for correctness.

How do the discriminative ($D$) neural nets initially know whether the data produced by $G$ is correct or not? Do we have to train the $D$ first then add it into the GAN with $G$?

Let's consider my trained $D$ net, which can classify a picture with 90% accuracy. If we add this $D$ to a GAN, there is a 10% probability it will classify an image wrong. If we train a GAN with this $D$, then will it also have the same 10% error in classifying an image? If yes, then why do GANs show promising results?

Answer 1

Compare generated and real data

All the results produced by G are always considered "wrong" by definition, even for a very good generator.

You provide the discriminative neural network $D$ with a mix of results generated by the generator network $G$ and real results from an outside source, and then you train it to distinguish if the result was produced by the generator or not - you're not comparing "good" and "bad" results, you're comparing real versus generated results.

This will result in a "mutual evolution" as $D$ will learn to find features that separate real results from generated ones, and $G$ will learn how to generate results that are hard to distinguish from real data.

Answer 2

A discriminative network ($D$) learns to discriminate by definition - we provide it with the true and the generated data, and let it learn by itself how to discriminate between the two.

Therefore, we expect network $D$ to improve the ability of network $G$ to generate better and better images (or other kind of data), as it try to "trick" network $D$ by producing new data that is more similar to "real data". It is not about the accuracy of network $D$ at all. It is not about improving the accuracy, it is about improving the ability of the computer to generate more "believable" data.

That said, using this scenario could be a good "unsupervised" way to improve the classification power of neural networks, as it forces the generator model to learn better features of real data, and to learn how to distinguish between actual features and noise, using much less data that is needed for a traditional supervised learning scheme.