How can the discriminator determine the sample is fake or real?

Question

Based on the articles I've read, the discriminator can identify whether a sample is fake or real. However, the articles don't clarify the conditions used to determine if a sample is fake or real. I came across information suggesting that labeling all the samples generated by the generator as fake before the training process, but I'm not sure if this is correct. Could someone please provide clarification? Thanks

Answer 1

In a GAN, the discriminator starts untrained, and the generator and discriminator are trained alongside each other. The process relies on neither being too strong for the other at any one stage, so that training can continue.

So the reason the discriminator can determine the difference between a real or fake sample is because it has learned from the GAN training process.

You should start with some training data, in the form of real images that you want the generator to produce similar output to.

When you train the discriminator:

• For every real image you show it, the target label is "real" (for a simple binary discrimintor the single target output is $p(real) = 1$)

• For every generated image you show it, the target label is "fake" (e.g. $p(real) = 0$ for a binary class prediction)

• In every minibatch you should train the discriminator on a mix of real and fake images, labelled accordingly.

When you train the generator:

• You still show the generator outputs to the discriminator, but now your target is to make the generator better, so you lie to the discriminator and label the generator's image as "real" ($p(real) = 1$). The trick here is that you only do this to get the gradients at the generator output layer. The discriminator is not updated for these training examples, only the generator is.

There are a few variations of this process, but the core idea is always similar, and to answer your main question, the discriminator determines real or fake because this is part of the training process. It is very bad at its job at the start, and you need that to be the case.