9
$\begingroup$

I observed in several papers that the variational autoencoder's output is blurred, while GANs output is crisp and has sharp edges.

Can someone please give some intuition why that is the case? I did think a lot but couldn't find any logic.

Improve this question
$\endgroup$
0

3 Answers 3

Reset to default
4
$\begingroup$

The key is: VAE usually use a small latent dimension, the information of input is so hard to pass through this bottleneck, meanwhile it tries to minimize the loss with the batch of input data, you should know the result -- VAE can only have a mean and blurry output.

If you increase the bandwidth of the bottleneck, i.e. the size of latent vector, VAE can get a high reconstruction quality, e.g. Spatial-Z-VAE

Improve this answer
$\endgroup$
1
  • 4
    $\begingroup$ I don't understand why this answer is correct. GAN also requires a low-dimensional vector to generate images. $\endgroup$
    – nbro
    Dec 24, 2021 at 8:21
2
$\begingroup$

In essence, Variational Autoencoders learn an "explicit" distribution of the data by trying to fit the data via a multi-dimensional Gaussian/Normal distribution.

However, Generative Adversarial Networks learn an "implicit" distribution of data meaning that you cannot directly sample them.

Also, due to the deterministic nature of neural networks, GANs tend to learn a Dirac Delta function. If you're lucky and the training of the GAN is successful, you can therefore get sharper images, since the model doesn't have to explicitly deal with the noise injected into it due to samplings, hence this could be a simpler learning problem.

By deterministic, I mean assuming that you have no sampling anywhere in the middle layers of your model and only use the neural network as an input-output mapping function.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Can you share some sources for your statement "deterministic nature of neural networks GANs tend to learn a Dirac Delta function". It is quite interesting and I would like to look more about it $\endgroup$
    – Trect
    Nov 10, 2018 at 9:11
  • $\begingroup$ Nice comparison of VAE and GAN. I understand that GAN is trained differently, but what about blurriness of the output of Variational Autoencoder? Why is it so hard to VAE to produce sharper images? $\endgroup$
    – samu
    Jan 27, 2019 at 17:06
  • $\begingroup$ @samutamm, as it is mentioned, VAE learnt the distribution of the data by fitting it into a multi-dimension normal distribution. This results at having the generated samples from VAE models to be blurry because of the independence assumption of the samples given latent variables. Ref: Page 1 in openreview.net/pdf?id=B1ElR4cgg "...suffer from a well-recognized issue of the maximum likelihood training paradigm when combined with a conditional independence assumption on the output given the latent variables: they tend to distribute probability mass diffusely over the data space.." $\endgroup$
    – Yasmin
    Mar 16, 2019 at 19:49
1
$\begingroup$

The reason is because of L1 (or L2) reconstruction loss used in VAEs. As is discussed in Image-to-Image Translation with Conditional Adversarial Networks: " If we take a naive approach and ask the CNN to minimize the Euclidean distance between predicted and ground truth pix- els, it will tend to produce blurry results. This is because Euclidean distance is minimized by averaging all plausible outputs, which causes blurring". Further: " GANs learn a loss that tries to classify if the output image is real or fake, while simultaneously training a generative model to minimize this loss. Blurry images will not be tolerated since they look obviously fake."

For further details read the ablation study in 4.2 of linked paper.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .