How exactly does adversarial training help in handling mode-collapse in generative networks?

Question

Of my understanding mode-collapse is when there happen to be multiple classes in the dataset and the generative network converges to only one of these classes and generates images only within this class. On training the model more, the model converges to another class.

In Goodfellows NeurIPS presentation he clearly addressed how training a generative network in an adversarial manner avoids mode-collapse. How exactly do GAN's avoid mode-collapse? and did previous works on generative networks not try to address this?

Apart from the obvious superior performance (generally), is the fact that GAN's address mode-collapse make them far preferred over other ways of training a generative model?

Answer 1

I don't think he said that at all. Going back to the talk you'll see he mentions mode collapse comes from the naivete of using alternating gradient-based optimization steps because then $min_{\phi}max_{\theta}L(G_\phi, D_\theta)$ starts to look a lot like $max_{\theta}min_{\phi}L(G_\phi, D_\theta)$.

This is problematic because in the latter case the generator has an obvious minimum of transforming all generated output into a single-mode that the discriminator has considered acceptable.

Since then a lot of work has been done to deal with this point of failure. Examples include Unrolled GANs (he mentions this one in the talk), where you essentially make the generator optimize what the discriminator will think $K$ steps in the future to ensure the ordering of the $min \ max$ game, and Wasserstein GANs, where you focus on a different metric that still has the same global minimum but allows for side by side training completely eliminating the ordering and failure mode, to begin with. On top of this, other work has been done as well, these are just two important examples.

Regarding how they fare against other generative models, like VAEs, there is no one is better than the other. The recent empirical success of GANs is why they are so popularly used, but we still see others being used in practice as well.