Questions tagged [variational-autoencoder]

For questions related to variational auto-encoders (VAEs). The first VAE was proposed in "Auto-Encoding Variational Bayes" (2013) by Diederik P. Kingma and Max Welling. There are several other VAEs, for example, the conditional VAE.

Filter by
Sorted by
Tagged with
2
votes
1answer
52 views

Why does KL divergence not satisfy the triangle inequality?

$D_{KL}=\sum_i p(x_i)log(p(x_i)/q(x_i)$ Also can't you make it satisfy the triangle inequality by taking the absolute value of the information at every point?
4
votes
0answers
74 views

Why can't VAE do sequence to sequence name generation?

I'm working on research in this sector where my supervisor wants to do cannonicalization of name data using VAEs, but I don't think it's possible to do, but I don't know explicitly how to show it ...
1
vote
1answer
28 views

In $logp_{\theta}(x^1,…,x^N)=D_{KL}(q_{\theta}(z|x^i)||p_{\phi}(z|x^i))+\mathbb{L}(\phi,\theta;x^i)$ why is $\theta$ and param for $p$ and $q$?

In $logp_{\theta}(x^1,...,x^N)=D_{KL}(q_{\theta}(z|x^i)||p_{\phi}(z|x^i))+\mathbb{L}(\phi,\theta;x^i)$ why is $\theta$ and param for $p$ and $q$? Why does $p(x^1,...,x^N)$ and $q(z|x^i)$ have the ...
2
votes
1answer
53 views

Why is the evidence equal to the KL divergence plus the loss?

Why is the equation $logp_{\theta}(x^1,...,x^N)=D_{KL}(q_{\theta}(z|x^i)||p_{\phi}(z|x^i))+\mathbb{L}(\phi,\theta;x^i)$ true, where $x^i$ are data points and $z$ are latent variables I was reading ...
1
vote
1answer
27 views

Why is exp used in encoder of VAE instead of using the value of standard deviation alone?

There's one VAE example here: https://towardsdatascience.com/teaching-a-variational-autoencoder-vae-to-draw-mnist-characters-978675c95776. And the source code of encoder can be found at the ...
1
vote
1answer
23 views

What is the 'Mean' in Variational Auto-encoder

Here's an example of Variational Auto-Encoder (VAE): There are 2 nodes before the Sample (encoding vector). One is 'Mean', one is 'Standard Deviation', the 'Mean' one is confusing. Is it Mean of ...
1
vote
0answers
25 views

Is Gradient Descent algorithm a part of Calculus of Variations?

As in https://en.wikipedia.org/wiki/Calculus_of_variations The calculus of variations is a field of mathematical analysis that uses variations, which are small changes in functions and ...
0
votes
0answers
17 views

What is the need for Auxiliary Decoder in the VAE-GAN?

The below image taken from Tim Sainberg's GitHub repo (https://github.com/timsainb) shows the structure of a VAE-GAN: My question is about the second row in the diagram. Random samples drawn from z ...
1
vote
0answers
19 views

What's the difference between semi-supervised VAEs and conditional VAEs?

Can someone explain the difference? I'm assuming the difference is just that the neural nets representing the encoder and decoder are trained in a semi-supervised manner in semi-supervised VAE, which ...
1
vote
0answers
21 views

Why is my variational auto-encoder generating random noise?

This is my first variational autoencoder. Background info: I am using the MNIST digits dataset. The model is created and trained in PyTorch. The model is able to get a reasonably low loss, but the ...
2
votes
0answers
18 views

What is the input for the prior model of VQ-VAE?

I'm trying to implement the VQ-VAE model. In there, a continuous variable $x$ is encoded in an array $z$ of discrete latent variables $z_i$ that are mapped each to an embedding vector $e_i$. These ...
2
votes
0answers
20 views

VAE latent space collapse

I'm trying to train a VAE using a graph dataset. However, my latent space shrinks epoch by epoch. Meanwhile, my ELBO plot comes to a steady state after few epochs. I tried to play around with ...
2
votes
2answers
61 views

What makes GAN or VAE better at image generation than NN that directly maps inputs to images

Say a simple neural network's input is a collection of tags (encoded in some way), and the output is an image that corresponds to those tags. Say this network consists of some dense layers and some ...
3
votes
0answers
26 views

In variational auto-encoders, what should the dimension of the mean and variance of the latent distribution and the latent vector be?

I'm having a problem to understand the needed dimensions of an VAE, especially for mu, logvar and z layer. Let's say I have an input of 512x512, 1 color channel (CT images), batch size 32. Then my ...
5
votes
1answer
65 views

Concrete example of latent variables and observables plugged into the Bayes' rule

In the context of the variational auto-encoder, can someone give me a concrete example of the application of the Bayes' rule $$p_{\theta}(z|x)=\frac{p_{\theta}(x|z)p(z)}{p(x)}$$ for a given latent ...
3
votes
0answers
39 views

Does MMD-VAE solve the problem of blurred images of vanilla VAEs?

I understand that with vanilla VAEs, there are a few reasons justifying the production of blurred out images. The InfoVAE paper describes the case when the decoder is flexible enough to ignore the ...
0
votes
0answers
22 views

Reduce same sample distance in VAE encodings

I'm working on a beta VAE model learning a latent representation used as a similarity metric for image registration. One of the main problems I'm facing is that the encoder + sampler output doesn't ...
1
vote
0answers
26 views

Are there any general tips for troubleshooting a VAE when apparently it is not learning?

I am trying to train a VAE for anomaly detection. I chose one architecture from this Github repository and I adjusted the input and output to match what I need. In my case, the input (and hence the ...
2
votes
1answer
40 views

Kullback-Leibler Divergence? How does it give knowledge gained?

I'm reading on KL Divergence on Wikipedia and I don't understand how the equation gives "information gained" as it says. I was under the impression that KL divergence is a way of measure difference ...
3
votes
2answers
70 views

What's going on in the equation of the variational lower bound?

I don't really understand what this equation is saying or what the purpose of the ELBO is? How does it help us find the true posterior distribution?
6
votes
1answer
317 views

How should we choose the dimensions of the encoding layer in auto-encoders?

How should we choose the dimensions of the encoding layer in auto-encoders?
2
votes
1answer
1k views

Why doesn't VAE suffer mode collapse?

Mode collapse is a common problem faced by GANs. I am curious why doesn't VAE suffer mode collapse?