12
$\begingroup$

In the paper Progressive growing of gans for improved quality, stability, and variation (ICLR, 2018) by Nvidia researchers, the authors write

Furthermore, we observe that mode collapses traditionally plaguing GANs tend to happen very quickly, over the course of a dozen minibatches. Commonly they start when the discriminator overshoots, leading to exaggerated gradients, and an unhealthy competition follows where the signal magnitudes escalate in both networks. We propose a mechanism to stop the generator from participating in such escalation, overcoming the issue (Section 4.2)

What do they mean by "the discriminator overshoots" and "the signal magnitudes escalate in both networks"?

My current intuition is that the discriminator gets too good too soon, which causes the generator to spike and try to play catch up. That would be the unhealthy competition that they are talking about. Model collapse is the side effect where the generator has trouble playing catch up and decides to play it safe by generating slightly varied images to increase its accuracy. Is this way of interpreting the above paragraph correct?

Improve this question
$\endgroup$
1
  • $\begingroup$ Note that it's not "Model collapse" (as you write) but "mode collapse". This refers to the mode of a distribution (if I remember correctly). $\endgroup$
    – nbro
    Jun 26 at 9:00

2 Answers 2

Reset to default
0
$\begingroup$

A discriminator overshooting may result in a dataset that has not been thoroughly clean and probably has too many identical feature, as a result there will be an early convergence from the discriminator as there is little variation. The drawback from this is that the model will not bee able to generalize well.

Improve this answer
$\endgroup$
1
  • $\begingroup$ In other words, the generator can no longer effective learn how to fool the discriminator because everything fails. $\endgroup$
    – David Hoelzer
    Mar 5, 2020 at 14:40
0
$\begingroup$

Yes, your intuition is correct. The effect of this problem is that the generator can no longer improve its output to marginally fool the discriminator - the discriminator isn't buying any of the generated output. In this case, the generator gets stuck in a local minimum and typically produces nonsense results.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .