Since the discriminator defines how the generator is updated, then building a discriminator with a higher number of parameters/more layers should lead to a better quality of generated samples. So, assuming that it won't lead to overwhelming the generator (discriminator loss toward 0) or mode collapse, when engineering a GAN, I should build a discriminator as good as possible?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
1
If you start to train a GAN in a way in which the discriminator becomes very powerful than the generator, then the training of generator will not be very much successful. In the same way, if the discriminator is too low in comparison with generator, it would let it produce any thing. So, either way, it will affect the training of the entire system.
So, if you see here, both generator and discriminator and competing against each other and on the other side they are dependent on each other for efficient training.
But I've read in a research paper that adding noise to discriminator is good for the overall stability. And sometimes people also change the labelling system as well. For instance, the value for YES is 1 and people change it to 0.9. This actually keeps the discriminator from being over confident.
-
$\begingroup$ It is only true for the original GAN loss. The SOTA approaches use different loss functions and regularization (WGAN-GP, R1 penalty) $\endgroup$ Commented May 13, 2022 at 10:48