I've been doing some reading about GANs, and although I've seen several excellent examples of implementations, the descriptions of why those patterns were chosen isn't clear to me in many cases.
At a very high level, the purpose of the discriminator in a GAN is establish a loss function that can be used to train the generator.
ie. Given the random input to the generator, the discriminator should be able to return a probability of the result being a 'real' image.
If the discriminator is perfect the probability will always be zero, and the loss function will have no gradient.
Therefore you iterate:
- generate random samples
- generate output from the generator
- evaluate the output using the discriminator
- train the generator
- update the discriminator to be more accurate by training it on samples from the real distribution and output from the generator.
The problem, and what I don't understand, is point 5 in the above.
Why do you use the output of the generator?
I absolutely understand that you need to iterate on the accuracy of the discriminator.
To start with it needs to respond with a non-zero value for the effectively random output from the generator, and slowly it needs to converge towards correctly classifying images at 'real' or 'fake'.
In order to achieve this we iterate, training the generator with images from the real distribution, pushing it towards accepting 'real' images.
...and with the images from the generator; but I don't understand why.
Effectively, you have a set of real images (eg. 5000 pictures of faces), that represent a sample from the latent space you want the GAN to converge on (eg. all pictures of faces).
So the argument goes:
As the generator is trained iteratively closer and closer to generating images from the latent space, the discriminator is iteratively trained to recognise from the latent space, as though it had a much larger sample size than the 5000 (or whatever) sample images you started with.
...ok, but that's daft.
The whole point of DNN's is that given a sample you can train it to recognise images from the latent space the samples represent.
I've never seen a DNN where the first step was 'augment your samples with extra procedurally generated fake samples'; the only reason to do this would be if you can only recognise samples in the input set, ie. your network is over-fitted.
So, as a specific example, why can't you incrementally train the discriminator on samples of ('real' * epoch/iterations + 'noise' * 1 - epoch/iterations), where 'noise' is just a random input vector.
Your discriminator will then necessarily converge towards recognising real images, as well as offering a meaningful gradient to the generator.
What benefit does feeding the output of the generator in offer over this?