9
$\begingroup$

I observed in several papers that the variational autoencoder's output is blurred, while GANs output is crisp and has sharp edges.

Can someone please give some intuition why that is the case? I did think a lot but couldn't find any logic.

Improve this question
$\endgroup$
0

5 Answers 5

Reset to default
4
$\begingroup$

The key is: VAE usually use a small latent dimension, the information of input is so hard to pass through this bottleneck, meanwhile it tries to minimize the loss with the batch of input data, you should know the result -- VAE can only have a mean and blurry output.

If you increase the bandwidth of the bottleneck, i.e. the size of latent vector, VAE can get a high reconstruction quality, e.g. Spatial-Z-VAE

Improve this answer
$\endgroup$
1
  • 9
    $\begingroup$ I don't understand why this answer is correct. GAN also requires a low-dimensional vector to generate images. $\endgroup$
    – nbro
    Commented Dec 24, 2021 at 8:21
2
$\begingroup$

In essence, Variational Autoencoders learn an "explicit" distribution of the data by trying to fit the data via a multi-dimensional Gaussian/Normal distribution.

However, Generative Adversarial Networks learn an "implicit" distribution of data meaning that you cannot directly sample them.

Also, due to the deterministic nature of neural networks, GANs tend to learn a Dirac Delta function. If you're lucky and the training of the GAN is successful, you can therefore get sharper images, since the model doesn't have to explicitly deal with the noise injected into it due to samplings, hence this could be a simpler learning problem.

By deterministic, I mean assuming that you have no sampling anywhere in the middle layers of your model and only use the neural network as an input-output mapping function.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Can you share some sources for your statement "deterministic nature of neural networks GANs tend to learn a Dirac Delta function". It is quite interesting and I would like to look more about it $\endgroup$
    – Trect
    Commented Nov 10, 2018 at 9:11
  • $\begingroup$ Nice comparison of VAE and GAN. I understand that GAN is trained differently, but what about blurriness of the output of Variational Autoencoder? Why is it so hard to VAE to produce sharper images? $\endgroup$
    – samu
    Commented Jan 27, 2019 at 17:06
  • $\begingroup$ @samutamm, as it is mentioned, VAE learnt the distribution of the data by fitting it into a multi-dimension normal distribution. This results at having the generated samples from VAE models to be blurry because of the independence assumption of the samples given latent variables. Ref: Page 1 in openreview.net/pdf?id=B1ElR4cgg "...suffer from a well-recognized issue of the maximum likelihood training paradigm when combined with a conditional independence assumption on the output given the latent variables: they tend to distribute probability mass diffusely over the data space.." $\endgroup$
    – Yasmin
    Commented Mar 16, 2019 at 19:49
1
$\begingroup$

The reason is because of L1 (or L2) reconstruction loss used in VAEs. As is discussed in Image-to-Image Translation with Conditional Adversarial Networks: " If we take a naive approach and ask the CNN to minimize the Euclidean distance between predicted and ground truth pix- els, it will tend to produce blurry results. This is because Euclidean distance is minimized by averaging all plausible outputs, which causes blurring". Further: " GANs learn a loss that tries to classify if the output image is real or fake, while simultaneously training a generative model to minimize this loss. Blurry images will not be tolerated since they look obviously fake."

For further details read the ablation study in 4.2 of linked paper.

Improve this answer
$\endgroup$
0
$\begingroup$

The reason is simple: because VAE's & GANs are almost opposite in their strengths and weaknesses.

VAE's (even VQ-VAEs!) always introduce some kind of noise to their encoder's output which has two consequences: it makes the latent space interoperable which increases generation diversity (the goal), but the added noise introduces some error/blur (unintended side-effect).

In contrast GAN's always uses adversaries to detect bad (e.g. blurry) images which has two consequences: the generated fake images are nearly indistinguishable from real images (the goal), the generator tends drop modes (aka reduce diversity) to make fooling the adversary easier (unintended side effect).

P.S. for VQ-VAEs: The noise in introduced by the vector-quantization & limited codebook size which causes a many-to-one (i.e. noisy) mapping.

Improve this answer
$\endgroup$
0
$\begingroup$

Fundamentally speaking, the optimization target of a VAE has two parts: minimizing reconstruction loss (similar to an AutoEncoder) and minimizing KL divergence loss (ensuring that the latent z follows a normal distribution so we can sample from it). However, these two objectives are antagonistic. The former aims to clearly distinguish different classes in the latent space to facilitate the final reconstruction, while the latter aims to mix different classes in the latent space to conform to a standard normal distribution. Therefore, after optimization, the images generated by a VAE are unlikely to be very clear and sharp.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .