# Tag Info

### Why doesn't VAE suffer mode collapse?

With Generative Adversarial Networks, all the generator cares about is fooling the discriminator. There's no requirement to be clever, or exhaustive, or make efficient use of the input space. As long ...
• 308
Accepted

• 40.9k

### What exactly is meant by variational distribution?

The variational distribution is the distribution (or set of distributions) that you use to approximate the distribution you are looking for. It's often denoted by $q$, $q_\phi$ or $q_\phi(z \mid x)$, ...
• 40.9k
Accepted

### In the VAE, why is $z \sim \mathcal{N}(\mu, \sigma^2)$ equivalent to $z = \mu + \sigma \odot \epsilon$?

I'll attempt a less formal explanation. The distribution $\mathcal{N}(\mu, \sigma)$ represents a normal distribution with mean $\mu$ and standard deviation $\sigma$. When we sample from this ...
• 2,531
Accepted

### Why do we regularize the variational autoencoder with a normal distribution?

If you are mathematically inclined, here is an article that discusses the reasoning. What I get as a take away is that the VAE forces the learned latent space to be Gaussian due to the KL divergence ...
• 724
Accepted

### Why does the KL divergence not satisfy the triangle inequality?

To prove that the KL divergence does not satisfy the triangle inequality, you just need a counterexample. Definitions KL divergence Let's first recapitulate the definition of KL divergence for ...
• 40.9k
Accepted

### How does the Kullback-Leibler divergence give "knowledge gained"?

You can know it better, if you know the concept of entropy: Information entropy is the average rate at which information is produced by a stochastic source of data. The information content (also ...
• 1,826

### How should we choose the dimensions of the encoding layer in auto-encoders?

The number of dimensions is a hyperparameter of your model, and you should do a hyperparameter search, like with any other parameters. There's also a tradeoff between dimension and training speed, so ...

### What's going on in the equation of the variational lower bound?

From this document, as you found here, $X$ is an observed variable and $Z$ is a hidden variable; $p(X)$ is the density function of $X$. The posterior distribution of the hidden variables can then be ...
• 1,826
Accepted

### In variational autoencoders, what does p(x|z) mean?

Whilst you're right that for any continuous distribution $P(X = x) = 0 \;; \forall x \in \mathcal{X}$ where $\mathcal{X}$ is there support of the distribution, they are not referring to probabilities ...
• 4,920

### In variational autoencoders, why do people use MSE for the loss?

On page 5 of the VAE paper, it's clearly stated We let $p_{\boldsymbol{\theta}}(\mathbf{x} \mid \mathbf{z})$ be a multivariate Gaussian (in case of real-valued data) or Bernoulli (in case of binary ...
• 40.9k
Accepted

### Why don't we also need to approximate $p(x \mid z)$ in the VAE?

What I can guess here is that, in VAEs, we assume $p(z)$ (prior), so we are able to calculate $p(x \mid z)$, but for $p(x)$ we can't assume its distribution? Is it right? You could assume $p(x)$ is ...
• 40.9k

### How to generate new data given a trained VAE - sample from the learned latent space or from multivariate Gaussian?

Few more clarifications. While the correct thing to do is draw from the prior, we have no guarantees that the aggregated posterior will cover the prior. Think of the aggregated posterior as the ...
• 316

### Comparison of the two alternative forms for the KL divergence

The KL divergence is just not symmetric, and so changing $q$ for $p$, and vice-versa, gives you a different behavior because the expectation is computed on a different distribution. In the first plot,...
• 2,958

### How to expand reconstruction error to mean squared error in Variational AutoEncoder?

In a way, you're right. The reconstruction loss is just an idea because you have not yet defined the distribution $p_\theta$. If you assume that this distribution is e.g. a Gaussian, then you should ...
• 40.9k
Accepted

### How to perform latent space Interpolation between two images?

You need more training images. Far more, at least a few hundred, with variations. The latent space has no meaningful form to it when you train with just two end points. The decoder will have no ...
• 32.7k