All Questions
Tagged with vae or variational-autoencoder
11 questions
7
votes
2
answers
3k
views
How is this Pytorch expression equivalent to the KL divergence?
I found the following PyTorch code (from this link)
-0.5 * torch.sum(1 + sigma - mu.pow(2) - sigma.exp())
where mu is the mean ...
- 805
9
votes
5
answers
13k
views
Why is the variational auto-encoder's output blurred, while GANs output is crisp and has sharp edges?
I observed in several papers that the variational autoencoder's output is blurred, while GANs output is crisp and has sharp edges.
Can someone please give some intuition why that is the case? I did ...
- 269
6
votes
1
answer
1k
views
Why is the evidence equal to the KL divergence plus the loss?
Why is the equation $$\log p_{\theta}(x^1,...,x^N)=D_{KL}(q_{\theta}(z|x^i)||p_{\phi}(z|x^i))+\mathbb{L}(\phi,\theta;x^i)$$ true, where $x^i$ are data points and $z$ are latent variables?
I was ...
- 805
4
votes
1
answer
1k
views
How does the VAE learn a joint distribution?
I found the following paragraph from An Introduction to
Variational Autoencoders sounds relevant, but I am not fully understanding it.
A VAE learns stochastic mappings between an observed $\mathbf{x}$...
- 243
8
votes
3
answers
2k
views
How does backprop work through the random sampling layer in a variational autoencoder?
Implementations of variational autoencoders that I've looked at all include a sampling layer as the last layer of the encoder block. The encoder learns to generate a mean and standard deviation for ...
- 183
7
votes
1
answer
7k
views
Why doesn't VAE suffer mode collapse?
Mode collapse is a common problem faced by GANs. I am curious why doesn't VAE suffer mode collapse?
- 269
3
votes
2
answers
9k
views
What is an appropriate size for a latent space of (variational) autoencoders and how it varies with the features of the images?
I am training an autoencoder and a variational autoencoder using satellite and streetview images. I have tested my program on standard datasets such as MNIST and CelebA. It seems that the latent space ...
- 33
2
votes
2
answers
2k
views
What is the advantage of using a VAE over a deterministic auto-encoder?
What is the advantage of using a VAE over a deterministic auto-encoder?
For example, assuming we have just 2 labels, a deterministic auto-encoder will always map a given image to the same latent ...
- 235
2
votes
1
answer
2k
views
How does the implementation of the VAE's objective function equate to ELBO?
For a lot of VAE implementations I've seen in code, it's not really obvious to me how it equates to ELBO.
$$L(X)=H(Q)-H(Q:P(X,Z))=\sum_ZQ(Z)logP(Z,X)-\sum_ZQ(Z)log(Q(Z))$$
The above is the definition ...
- 805
1
vote
0
answers
119
views
Is VAE the same as the E-step of the EM algorithm?
EM(Expectation Maximum)
Target: maximize $p_\theta(x)$
$ p_\theta(x)=\frac{p_\theta(x, z)}{p_\theta(z \mid x)} \\\\$
Take log on both sides:
$ \log p_\theta(x)=\log p_\theta(x, z)-\log p_\theta(z \...
- 11
1
vote
1
answer
218
views
Trying to understand some derivation in the paper: Deep Unsupervised Learning using Nonequilibrium Thermodynamics
I have recently been learning about diffusion models and trying to derive all the results in the paper by Sohl-Dickstein, et. al, "Deep Unsupervised Learning using Nonequilibrium Thermodynamics&...
- 31