# Questions tagged [variational-autoencoder]

For questions related to variational auto-encoders (VAEs). The first VAE was proposed in "Auto-Encoding Variational Bayes" (2013) by Diederik P. Kingma and Max Welling. There are several other VAEs, for example, the conditional VAE.

22 questions
Filter by
Sorted by
Tagged with
52 views

### Why does KL divergence not satisfy the triangle inequality?

$D_{KL}=\sum_i p(x_i)log(p(x_i)/q(x_i)$ Also can't you make it satisfy the triangle inequality by taking the absolute value of the information at every point?
74 views

### Why can't VAE do sequence to sequence name generation?

I'm working on research in this sector where my supervisor wants to do cannonicalization of name data using VAEs, but I don't think it's possible to do, but I don't know explicitly how to show it ...
28 views

### In $logp_{\theta}(x^1,…,x^N)=D_{KL}(q_{\theta}(z|x^i)||p_{\phi}(z|x^i))+\mathbb{L}(\phi,\theta;x^i)$ why is $\theta$ and param for $p$ and $q$?

In $logp_{\theta}(x^1,...,x^N)=D_{KL}(q_{\theta}(z|x^i)||p_{\phi}(z|x^i))+\mathbb{L}(\phi,\theta;x^i)$ why is $\theta$ and param for $p$ and $q$? Why does $p(x^1,...,x^N)$ and $q(z|x^i)$ have the ...
53 views

### Why is the evidence equal to the KL divergence plus the loss?

Why is the equation $logp_{\theta}(x^1,...,x^N)=D_{KL}(q_{\theta}(z|x^i)||p_{\phi}(z|x^i))+\mathbb{L}(\phi,\theta;x^i)$ true, where $x^i$ are data points and $z$ are latent variables I was reading ...
27 views

### Why is exp used in encoder of VAE instead of using the value of standard deviation alone?

There's one VAE example here: https://towardsdatascience.com/teaching-a-variational-autoencoder-vae-to-draw-mnist-characters-978675c95776. And the source code of encoder can be found at the ...
23 views

### What is the 'Mean' in Variational Auto-encoder

Here's an example of Variational Auto-Encoder (VAE): There are 2 nodes before the Sample (encoding vector). One is 'Mean', one is 'Standard Deviation', the 'Mean' one is confusing. Is it Mean of ...
25 views

### Is Gradient Descent algorithm a part of Calculus of Variations?

As in https://en.wikipedia.org/wiki/Calculus_of_variations The calculus of variations is a field of mathematical analysis that uses variations, which are small changes in functions and ...
17 views

### What is the need for Auxiliary Decoder in the VAE-GAN?

The below image taken from Tim Sainberg's GitHub repo (https://github.com/timsainb) shows the structure of a VAE-GAN: My question is about the second row in the diagram. Random samples drawn from z ...
19 views

### What's the difference between semi-supervised VAEs and conditional VAEs?

Can someone explain the difference? I'm assuming the difference is just that the neural nets representing the encoder and decoder are trained in a semi-supervised manner in semi-supervised VAE, which ...
21 views

### Why is my variational auto-encoder generating random noise?

This is my first variational autoencoder. Background info: I am using the MNIST digits dataset. The model is created and trained in PyTorch. The model is able to get a reasonably low loss, but the ...
18 views

### What is the input for the prior model of VQ-VAE?

I'm trying to implement the VQ-VAE model. In there, a continuous variable $x$ is encoded in an array $z$ of discrete latent variables $z_i$ that are mapped each to an embedding vector $e_i$. These ...
20 views

### VAE latent space collapse

I'm trying to train a VAE using a graph dataset. However, my latent space shrinks epoch by epoch. Meanwhile, my ELBO plot comes to a steady state after few epochs. I tried to play around with ...
61 views

### What makes GAN or VAE better at image generation than NN that directly maps inputs to images

Say a simple neural network's input is a collection of tags (encoded in some way), and the output is an image that corresponds to those tags. Say this network consists of some dense layers and some ...
26 views

### In variational auto-encoders, what should the dimension of the mean and variance of the latent distribution and the latent vector be?

I'm having a problem to understand the needed dimensions of an VAE, especially for mu, logvar and z layer. Let's say I have an input of 512x512, 1 color channel (CT images), batch size 32. Then my ...
65 views

### Concrete example of latent variables and observables plugged into the Bayes' rule

In the context of the variational auto-encoder, can someone give me a concrete example of the application of the Bayes' rule $$p_{\theta}(z|x)=\frac{p_{\theta}(x|z)p(z)}{p(x)}$$ for a given latent ...
39 views

### Does MMD-VAE solve the problem of blurred images of vanilla VAEs?

I understand that with vanilla VAEs, there are a few reasons justifying the production of blurred out images. The InfoVAE paper describes the case when the decoder is flexible enough to ignore the ...
22 views

### Reduce same sample distance in VAE encodings

I'm working on a beta VAE model learning a latent representation used as a similarity metric for image registration. One of the main problems I'm facing is that the encoder + sampler output doesn't ...
26 views

### Are there any general tips for troubleshooting a VAE when apparently it is not learning?

I am trying to train a VAE for anomaly detection. I chose one architecture from this Github repository and I adjusted the input and output to match what I need. In my case, the input (and hence the ...
40 views

### Kullback-Leibler Divergence? How does it give knowledge gained?

I'm reading on KL Divergence on Wikipedia and I don't understand how the equation gives "information gained" as it says. I was under the impression that KL divergence is a way of measure difference ...
70 views

### What's going on in the equation of the variational lower bound?

I don't really understand what this equation is saying or what the purpose of the ELBO is? How does it help us find the true posterior distribution?