Questions tagged [variational-autoencoder]

For questions related to variational auto-encoders (VAEs). The first VAE was proposed in "Auto-Encoding Variational Bayes" (2013) by Diederik P. Kingma and Max Welling. There are several other VAEs, for example, the conditional VAE.

Filter by
Sorted by
Tagged with
7
votes
1answer
648 views

How should we choose the dimensions of the encoding layer in auto-encoders?

How should we choose the dimensions of the encoding layer in auto-encoders?
7
votes
1answer
931 views

What are the fundamental differences between VAE and GAN for image generation?

Starting from my own understanding, and scoped to the purpose of image generation, I'm well aware of the major architectural differences: A GAN's generator samples from a relatively low dimensional ...
5
votes
2answers
276 views

Why is “-0.5 * torch.sum(1 + sigma - mu.pow(2) - sigma.exp())” in Pytorch equivalent to the KL?

I found this code for the loss function of a VAE: -0.5 * torch.sum(1 + sigma - mu.pow(2) - sigma.exp()) From this link: https://debuggercafe.com/getting-started-...
5
votes
1answer
75 views

Concrete example of latent variables and observables plugged into the Bayes' rule

In the context of the variational auto-encoder, can someone give me a concrete example of the application of the Bayes' rule $$p_{\theta}(z|x)=\frac{p_{\theta}(x|z)p(z)}{p(x)}$$ for a given latent ...
4
votes
1answer
266 views

Why is the evidence equal to the KL divergence plus the loss?

Why is the equation $$\log p_{\theta}(x^1,...,x^N)=D_{KL}(q_{\theta}(z|x^i)||p_{\phi}(z|x^i))+\mathbb{L}(\phi,\theta;x^i)$$ true, where $x^i$ are data points and $z$ are latent variables? I was ...
4
votes
2answers
376 views

What's going on in the equation of the variational lower bound?

I don't really understand what this equation is saying or what the purpose of the ELBO is. How does it help us find the true posterior distribution?
4
votes
1answer
267 views

Does MMD-VAE solve the problem of blurred images of vanilla VAEs?

I understand that with vanilla VAEs, there are a few reasons justifying the production of blurred out images. The InfoVAE paper describes the case when the decoder is flexible enough to ignore the ...
4
votes
1answer
336 views

Is there a continuous conditional variational auto-encoder?

The Conditional Variational Autoencoder (CVAE), introduced in the paper Learning Structured Output Representation using Deep Conditional Generative Models (2015), is an extension of Variational ...
4
votes
0answers
132 views

Why can't VAE do sequence to sequence name generation?

I'm working on research in this sector where my supervisor wants to do cannonicalization of name data using VAEs, but I don't think it's possible to do, but I don't know explicitly how to show it ...
4
votes
1answer
388 views

What is the input for the prior model of VQ-VAE?

I'm trying to implement the VQ-VAE model. In there, a continuous variable $x$ is encoded in an array $z$ of discrete latent variables $z_i$ that are mapped each to an embedding vector $e_i$. These ...
3
votes
1answer
117 views

What does the notation $\mathcal{N}(z; \mu, \sigma)$ stand for in statistics?

I know that the notation $\mathcal{N}(\mu, \sigma)$ stands for a normal distribution. But I'm reading the book "An Introduction to Variational Autoencoders" and in it, there is this notation:...
3
votes
2answers
540 views

What makes GAN or VAE better at image generation than NN that directly maps inputs to images

Say a simple neural network's input is a collection of tags (encoded in some way), and the output is an image that corresponds to those tags. Say this network consists of some dense layers and some ...
3
votes
1answer
3k views

Why doesn't VAE suffer mode collapse?

Mode collapse is a common problem faced by GANs. I am curious why doesn't VAE suffer mode collapse?
3
votes
1answer
74 views

What does the approximate posterior on latent variables, $q_\phi(z|x)$, tend to when optimising VAE's

The ELBO objective is described as follows $$ ELBO(\phi,\theta) = E_{q_\phi(z|x)}[log p_\theta (x|z)] - KL[q_\phi (z|x)||p(z)] $$ This form of ELBO includes a regularisation term in the form of the ...
3
votes
1answer
76 views

How do you calculate KL divergence on a three-dimensional space for a Variational Autoencoder?

I'm trying to implement a variational auto-encoder (as seen in Section 3.1 here: https://arxiv.org/pdf/2004.06271.pdf). It differs from a traditional VAE because it encodes its input images to three-...
3
votes
1answer
227 views

Why does the variational auto-encoder use the reconstruction loss?

VAE is trained to reduce the following two losses. KL divergence between inferred latent distribution and Gaussian. the reconstruction loss I understand that the first one regularizes VAE to get ...
3
votes
2answers
566 views

How to use a VAE to reconstruct an image starting from an initial image instead of starting from a random vector?

Is it possible to use a VAE to reconstruct an image starting from an initial image instead of using K.random_normal, as shown in the “sampling” function of this ...
3
votes
0answers
45 views

In variational auto-encoders, what should the dimension of the mean and variance of the latent distribution and the latent vector be?

I'm having a problem to understand the needed dimensions of an VAE, especially for mu, logvar and z layer. Let's say I have an input of 512x512, 1 color channel (CT images), batch size 32. Then my ...
2
votes
1answer
95 views

How does the Kullback-Leibler divergence give “knowledge gained”?

I'm reading about the KL divergence on Wikipedia. I don't understand how the equation gives "information gained" as it says in the "Interpretations" section Expressed in the ...
2
votes
1answer
160 views

Why does KL divergence not satisfy the triangle inequality?

$D_{KL}=\sum_i p(x_i)log(p(x_i)/q(x_i)$ Also can't you make it satisfy the triangle inequality by taking the absolute value of the information at every point?
2
votes
1answer
41 views

What do they mean by “contradictory loss”?

In page 4 of the paper https://arxiv.org/pdf/2009.07047v1.pdf, it says the encoder $E_{R,X}$ of $VAE_1$ tries to fool the discriminator with a contradictory loss to ensure that $R$ and $X$ are mapped ...
2
votes
1answer
45 views

What is the mean in the variational auto-encoder?

Here's a diagram of a variational auto-encoder. There are 2 nodes before the sample (encoding vector). One is the mean, one is the standard deviation. The mean one is confusing. Is it the mean of ...
2
votes
0answers
27 views

Compressing Parameters of an Response System

I have an input-output system, which is fully determined by 256 parameters, of which I know a significant amount are of less importance to the input-output pattern. The data I have is some (64k in ...
2
votes
0answers
326 views

Why would a VAE train much better with batch sizes closer to 1 over batch size of 100+?

I've been training a VAE to reconstruct human names and when I train it on a batch size of 100+ after about 5 hours of training it tends to just output the same thing regardless of the input and I'm ...
2
votes
1answer
318 views

What is the impact of scaling the KL divergence and reconstruction loss in the VAE objective function?

Variational autoencoders have two components in their loss function. The first component is the reconstruction loss, which for image data, is the pixel-wise difference between the input image and ...
2
votes
0answers
17 views

Does bottleneck size matter in Disentangled Variational Autoencoders?

I suppose that picking an appropriate size for the bottleneck in Autoencoders is neither a trivial nor an intuitive task. After watching this video about VAEs, I've been wondering: Do disentangled ...
2
votes
0answers
49 views

Why does the ELBO come to a steady state and the latent space shrinks?

I'm trying to train a VAE using a graph dataset. However, my latent space shrinks epoch by epoch. Meanwhile, my ELBO plot comes to a steady state after a few epochs. I tried to play around with ...
2
votes
0answers
223 views

Are there any general tips for troubleshooting a VAE when apparently it is not learning?

I am trying to train a VAE for anomaly detection. I chose one architecture from this Github repository and I adjusted the input and output to match what I need. In my case, the input (and hence the ...
1
vote
2answers
70 views

In variational autoencoders, what does p(x|z) mean?

If $x \sim \mathcal{N}(\mu,\,\sigma^{2})$, then it is a continuous variable, and therefore $P(x) = 0$ for any x. One can only consider things like $P(x<X)$ to get a probability greater than 0. So ...
1
vote
1answer
102 views

Why is exp used in encoder of VAE instead of using the value of standard deviation alone?

There's one VAE example here: https://towardsdatascience.com/teaching-a-variational-autoencoder-vae-to-draw-mnist-characters-978675c95776. And the source code of encoder can be found at the ...
1
vote
1answer
32 views

Are there architectures to generate pictures from four labels? (VAEs, GANs)

I want to try something with image creation via NNs. I have come across Variational Autoencoders and Generative Adversarial Networks as possible solutions but have only found image creatinon with ...
1
vote
1answer
60 views

How does implementation of VAE's objective function equate to ELBO?

For a lot of VAE implementations I've seen in code, it's not really obvious to me how it equates to ELBO. $$L(X)=H(Q)-H(Q:P(X,Z))=\sum_ZQ(Z)logP(Z,X)-\sum_ZQ(Z)log(Q(Z))$$ The above is the definition ...
1
vote
1answer
79 views

How many types of variational auto-encoders are there?

I have been studying about auto-encoders and variational auto-encoders. I would like to know how many variants of VAEs are there today. If there are many variants, can they be used for feature ...
1
vote
1answer
43 views

What is the main contribution of the paper Disentangling by Factorising?

Considering the paper Disentangling by Factorising, in addition to introducing a new model for Disentangled Representation Learning, FactorVAE (see figure), what is the main theoretical contribution ...
1
vote
1answer
115 views

Can a variational auto-encoder learn images composed of random noise at each pixel (each drawn from the same distribution)?

Can a variational auto-encoder (VAE) learn images whose pixels have been generated from a Gaussian distribution (e.g. $N(0, 1)$), i.e. each pixel is a sample from $N(0, 1)$? My gut feeling says no, ...
1
vote
1answer
337 views

Why do we regularize the variational autoencoder with a normal distribution?

When we define the loss function of a variational autoencoder (VAE), we add the Kullback-Leibler divergence between the sample taken according to a normal distribution of parameters: $$ N(\mu,\sigma)...
1
vote
1answer
65 views

In this VAE formula, why do $p$ and $q$ have the same parameters?

In $$\log p_{\theta}(x^1,...,x^N)=D_{KL}(q_{\theta}(z|x^i)||p_{\phi}(z|x^i))+\mathbb{L}(\phi,\theta;x^i),$$ why does $p(x^1,...,x^N)$ and $q(z|x^i)$ have the same parameter $\theta?$ Given that $p$ is ...
1
vote
0answers
36 views

Doubts on how this Dimensions are handled (Tensorflow) and how mean and standard deviation are extracted [closed]

I'm following the introductory MIT Deep Learning course on Youtube and i've been stuck for a day now on this piece of code: ...
1
vote
0answers
101 views

Why does the VAE using a KL-divergence with a non-standard mean does not produce good images?

I know I can make a VAE do generation with a mean of 0 and std-dev of 1. I tested it with the following loss function: ...
1
vote
0answers
32 views

variational auto encoder loss goes down but does not reconstruct input. out of debugging ideas

My variational autoencoder seems to work for MNIST, but fails on slightly "harder" data. By "fails" I mean there are at least two apparent problems: Very poor reconstruction, for ...
1
vote
0answers
65 views

VAE giving near zero output when latent space dimension is large

I'm training a VAE to reconstruct some input (channels picked up by some MIMO BS for context) and I ran an experiment on the training set to see how the performance improves with the latent space ...
1
vote
0answers
82 views

Hairstyle Virtual Try On

I want to help people with cancer who are under chemotherapy, and generally people who have lost their hair to Virtually Try-On Toupees/Wigs on their head. VTO must support both the frontal and side ...
1
vote
0answers
56 views

How can I reconstruct sparse one-hot encodings using an RBM?

I am currently working with a categorical-binary RBM, where there are 50 categorical visible units and 25 binary hidden units. The categorical visible units are expressed in one-hot encoding format, ...
1
vote
1answer
57 views

How to construct input dependent convolutional filter?

I am constructing a convolutional variational autoencoder for images, starting out with mnist digits. Typically I would specify convolutional layers in the following way: ...
1
vote
0answers
101 views

Is Gradient Descent algorithm a part of Calculus of Variations?

As in https://en.wikipedia.org/wiki/Calculus_of_variations The calculus of variations is a field of mathematical analysis that uses variations, which are small changes in functions and ...
1
vote
0answers
39 views

What's the difference between semi-supervised VAEs and conditional VAEs?

Can someone explain the difference? I'm assuming the difference is just that the neural nets representing the encoder and decoder are trained in a semi-supervised manner in semi-supervised VAE, which ...
1
vote
0answers
76 views

Why is my variational auto-encoder generating random noise?

This is my first variational autoencoder. Background info: I am using the MNIST digits dataset. The model is created and trained in PyTorch. The model is able to get a reasonably low loss, but the ...
1
vote
0answers
29 views

Reduce same sample distance in VAE encodings

I'm working on a beta VAE model learning a latent representation used as a similarity metric for image registration. One of the main problems I'm facing is that the encoder + sampler output doesn't ...
0
votes
2answers
78 views

In variational autoencoders, why do people use MSE for the loss?

In VAEs, we try to maximize the ELBO $\mathbb(E_q log\ p(x|z) + D_{KL}(q(z|x), p(z))$), but I see that many implement the first term as MSE of the image and it's reconstruction. Is this mathematically ...
0
votes
0answers
31 views

Can I use the VAE for dimensionality reduction?

I'm doing a project that uses a clustering algorithm for the facial expression classification task. So, I use the output of the encoder in the VAE autoencoder for dimensionality reduction. However, I ...