Questions tagged [variational-autoencoder]

For questions related to variational auto-encoders (VAEs). The first VAE was proposed in "Auto-Encoding Variational Bayes" (2013) by Diederik P. Kingma and Max Welling. There are several other VAEs, for example, the conditional VAE.

Filter by
Sorted by
Tagged with
1 vote
0 answers
19 views

Toy dataset: Radial VAE

I'm evaluating disentanglement in toy datasets seeing as we have such little understanding of the phenomena. I'm using various tools from differential geometry. Now I want to train a VAE on the ...
John Miller's user avatar
1 vote
1 answer
49 views

Variational Autoencoder (VAE) Multiclass Interpolation

I'm working on a variational autoencoder (VAE) with 20 different classes in my training data. I've successfully trained the VAE and can sample from the latent space to generate data points. However, I ...
Chels Zou's user avatar
0 votes
0 answers
8 views

What is the detailed experimental setup for class-incremental continual image generation?

Do you condition the generative model (let's say, VAE) on the task identity or the class label or both? If I condition the VAE on both task identity and class label, then I have to provide both the ...
Homie98's user avatar
0 votes
1 answer
53 views

What exactly is meant by variational distribution?

What specifically does the term "variational distribution" refer to? The encoder of Variational autoencoder? Forward process of denoising diffusion probabilistic models? Images or latent ...
diffusion stable's user avatar
0 votes
1 answer
70 views

Confusion over taking gradients in Variational Autoencoders (VAE)

I am confused as to when to hold certain parameters constant in a VAE. I will explain with a concrete example. We can write $\operatorname{ELBO}(\phi, \theta) = \mathbb{E}_{q_{\phi}(z)}\left[\log \...
Joel's user avatar
  • 23
0 votes
0 answers
38 views

What is the motivation of VQ-VAE?

I don't understand the motivation because I read that the motivation for VAE was that: 'it could be shown that it is not meaningful to interpolate the latent space of regular auto-encoders' (my own ...
profPlum's user avatar
  • 360
2 votes
1 answer
61 views

Why can Variational Autoencoders (VAEs) approximate arbitrary distributions?

I am trying to reason to myself why is it that VAEs can approximate arbitrary probability distributions even though 𝑞𝜙(𝑧|𝑥) and 𝑝𝜃(𝑥|𝑧) are Gaussian. I understand that the parameters are ...
Joel's user avatar
  • 23
0 votes
2 answers
78 views

How to expand reconstruction error to mean squared error in Variational AutoEncoder? [closed]

How to expand reconstruction error to mean squared error when it is $\mathbb{E}_{z\sim q_{\phi}(z|x)}[\log p_\theta(x|z)]$? [reconstruction error] $\mathbb{E}_{z\sim q_{\phi}(z|x)}[\log p_\theta(x|z)]$...
diffusion stable's user avatar
1 vote
1 answer
53 views

Comparison of the two alternative forms for the KL divergence [closed]

On page 468 of 'Pattern Recognition and Machine Learning', what does 'the same variables given by the product of two independent univariate Gaussian distributions' mean? The PDF says, The green ...
diffusion stable's user avatar
0 votes
0 answers
28 views

KL divergence and sign [duplicate]

In the Auto-Encoding Variational Bayes paper, the formula for KL divergence is $$ \frac{1}{2} \sum \bigl (1 + \log(σ^2) - μ^2 - σ^2 \bigr) \space\space\space\space...(10)$$ , but the equation is $$- ...
diffusion stable's user avatar
1 vote
0 answers
47 views

Why is $p(x)=\int p(x,z) dz$ intractable for continuous $z$ in VAE?

In VAE we use the importance sampling trick to use $q_\phi(z|x)$ to help maximize $\log p_\theta(x)=\log \int p_\theta(x,z)dz\ge \int q_\phi(z|x)\log \frac{p_\theta(x,z)}{q_\phi(z|x)} dz$. Meanwhile, ...
hjenryin's user avatar
0 votes
0 answers
29 views

Is this a valid application of Autoencodeers/VAE?

I am trying to predict a spectrum (1D vector) from various scalar inputs which are known to be correlated. As the spectrum vector is very long (4000 points) it was suggested that I use dimensionality ...
Christopher McQueen's user avatar
1 vote
2 answers
52 views

How to apply backpropagation when one layer of the network is a call-only function (no gradient)?

I worked with Feed Forward Neural Network and VAE and understood backpropagation algorithm. Now I build a VAE network, one layer of it is a very complex vector-to-vector function $f(x)$ (a general '...
whitegreen's user avatar
0 votes
1 answer
52 views

Any suggestion on what I should try to get this cVAE model working for random generation of new molecules?

I am using a cVAE model to generate new structures of molecules. After successfully training the VAE model I am able to get proper reconstructions of the training set. While I am also able to generate ...
Formal_this's user avatar
0 votes
0 answers
36 views

Layer Questions regarding Bidirectional VAE (D3VAE)

I am currently trying to figure out how D3VAE are working, but I can't seem to understand the network architecture given. The paper can be found here: https://openreview.net/pdf?id=rG0jm74xtx The ...
Patrick Lehnen's user avatar
0 votes
0 answers
29 views

Can I use this method to improve the VAE model?

My VAE model is not learning well. The model's learning history shows that the reconstruction loss is as large as approximately 8000 or more, and the KL-Divergence loss is diverging, starting with an ...
KYH's user avatar
  • 17
0 votes
1 answer
31 views

How to convert my test data in the same dimensionality as my train data

I have trained a VAE with jpg images. My latent space dimension has 768 features and when plotting the latent space it looks like this: However, when I use the scikit learn tool LDA (Linear ...
Dude Rar's user avatar
1 vote
2 answers
108 views

Why optimise log p(x) rather than log p(x|z) in a Variational AutoEncoder?

Background The loss function in a Variational AutoEncoder is the Evidence Lower Bound (ELBO): $\mathbb{E}_q[log$ $p(x|z)] - KL[q(z)||p(z)]$ And has this inequality: $log$ $p(x) \ge \mathbb{E}_q[log$ $...
Titus Buckworth's user avatar
0 votes
0 answers
33 views

Must dataset strictly come from marginal distribution in VAE?

My question is what if the population of the dataset is another marginal distribution, but whose support covers the original marginal distribution $p(\mathbf{x})$, can we use VAE to infer this target ...
Magi Feeney's user avatar
0 votes
1 answer
44 views

Why is the variational lower bound is easier to compute than the original marginal distribution?

Why is the ELBO of $p(x)=\int p(x|z)p(z)\mathrm{d}z$ easier to compute/estimate than the expression itself? Can we compute this quantity itself through sampling in the same way? I understanding that ...
Hanhan Li's user avatar
  • 101
0 votes
0 answers
59 views

Why most of VAE implementations for image reconstruction use a deterministic decoder?

The decoder of VAE is a parameterized distribution $p_{\theta}(\mathbb{x} | \mathbb{z})$ by definition, from which we can sample an output $\hat{\mathbb{x}}$ with an input $\mathbb{x}$ and a sampled ...
Magi Feeney's user avatar
0 votes
0 answers
28 views

Mean and std in vaerational autoencoder

Are mean and standard deviation in variational autoencoders equal? If not, then why are both calculated in the same way?
Hossein Goodarzi's user avatar
0 votes
0 answers
86 views

Using a VQ-VAE encoder's output to condition another model

I've trained a VQ-VAE on images that I want to use to condition another model (latent diffusion). The shape of my encoder's output is (4, 256, 56, 56) and I'm getting it using: ...
jbm's user avatar
  • 101
-1 votes
1 answer
58 views

Combination of VAE with GAN

I am going to implement a lecture which it aims to generate new images. It uses a variational autoencoder to produce latent vector and then feed it to a gan network as input. My question is, in ...
Pedram Yazdipoor's user avatar
0 votes
1 answer
145 views

How to generate new data using VAE?

I have built the following function which takes as input some data and runs a VAE on them: ...
quant's user avatar
  • 101
1 vote
0 answers
32 views

Is VAE the same as the E-step of the EM algorithm?

EM(Expectation Maximum) Target: maximize $p_\theta(x)$ $ p_\theta(x)=\frac{p_\theta(x, z)}{p_\theta(z \mid x)} \\\\$ Take log on both sides: $ \log p_\theta(x)=\log p_\theta(x, z)-\log p_\theta(z \...
Garfield's user avatar
0 votes
0 answers
63 views

How to optimize ELBO(VAE's loss function)?

Suppose we've got the following formula: $\log p(x;\theta)=\mathbb{E}_{q(z|x;\phi)}[\log p(x,z;\theta)-\log q(z|x;\phi)]+KL(q(z|x;\phi)||p(z|x;\theta))\\ \geq \mathbb{E}_{q(z|x;\phi)}[\log p(x,z;\...
Garfield's user avatar
2 votes
0 answers
107 views

Why use z_mean to plot the latent space learned by a Variational Autoencoder?

In the Keras website, there is an example code of a Variational Autoencoder. At the end of such a page, there is an example code that plots the latent space learned from MNIST. The code is as follows: ...
AlexSC's user avatar
  • 149
1 vote
1 answer
32 views

What models/algorithms besides variational autoencoders can I use to transform a discrete input into a differentiable latent space?

Let's say I have a discrete input and want to transform it into a differentiable latent space. What models/algorithms besides variational autoencoders can I use?
postnubilaphoebus's user avatar
0 votes
0 answers
15 views

What is $p(Z)$ and what happens to the variational posterior $q(Z;X)$ during data synthesis (after training)?

From my understanding of inference problems, we want to compute the posterior $p(Z|X=D)$, for some observed dataset $D=(x^1, x^2,\dots,x^n)$ of $n$ independent observations, in order to "update&...
Venkat Krishnamohan's user avatar
0 votes
0 answers
126 views

VAE - Which loss to optimize for?

Regarding hyperparameter optimization for VAEs. Should you optimize for the reconstruction loss, or the complete ELBO (- KL divergence + reconstruction loss)? My thought is that it probably depends on ...
RolandSt's user avatar
2 votes
0 answers
738 views

Why VQ-VAE instead of VAE?

From the paper on VQ-VAE, it said that the vector quantized variational autoencoder (VQ-VAE), differs from VAEs in two key ways: the encoder network output discrete, rather than continuous, codecs ...
Nervous Hero's user avatar
2 votes
2 answers
2k views

What is an appropriate size for a latent space of (variational) autoencoders and how it varies with the features of the images?

I am training an autoencoder and a variational autoencoder using satellite and streetview images. I have tested my program on standard datasets such as MNIST and CelebA. It seems that the latent space ...
Qingyi Wang's user avatar
1 vote
0 answers
11 views

Is it possible for PixelCNN to tell us what it generates?

I coded PixelCNN with the help of Keras official website. Also, I read the paper. I can use PixelCNN, similar to a decoder or generator (to generate samples). My question is, "is it possible to ...
Pouyan's user avatar
  • 29
1 vote
0 answers
66 views

Why does importance sampling work with latent variable models?

Caveat: importance sampling doesn't actually work for variational auto-encoders, but the question makes sense regardless In "L4 Latent Variable Models (VAE) -- CS294-158-SP20 Deep Unsupervised ...
Foobar's user avatar
  • 151
0 votes
1 answer
101 views

Is the discriminator of a GAN network embedded in VAE?

From what I understand, a Generative Adversarial Network (GAN) is composed of an encoder (generator), some synthetic data (fake data) and a discriminator that will penalize any distinguishable real ...
Rhesus's user avatar
  • 3
1 vote
1 answer
93 views

Datasets input at model.fit produce unexpected results of training loss vs validation loss

Im trying to train a neural network (VAE) using tensorflow and Im getting different results based on the type of input in the model.fit. When I input arrays I get normal difference between the ...
user avatar
0 votes
2 answers
84 views

Why does the coding layer in a VAE have a range of values?

While reading the book, Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems, I read that VAEs using a sampling technique to ...
desert_ranger's user avatar
0 votes
1 answer
196 views

How does a VAE sample the coding layer?

I am reading the book, Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems and came across the following paragraph - You ...
desert_ranger's user avatar
4 votes
2 answers
2k views

How to generate new data given a trained VAE - sample from the learned latent space or from multivariate Gaussian?

To generate synthetic dataset using a trained VAE, there is confusion between two approaches: Use learned latent space: z = mu + (eps * log_var) to generate (...
Arun's user avatar
  • 225
2 votes
1 answer
267 views

Why don't we also need to approximate $p(x \mid z)$ in the VAE?

In the VAE, we approximate the probability distribution $p(z \mid x)$, where $z$ is the latent vector and $x$ is our data. The reason is that $p(z \mid x)$ becomes impossible to calculate for ...
Nervous Hero's user avatar
3 votes
1 answer
101 views

Why do we use the same parameters for the joint, marginal and conditional distributions in VAEs?

I've noticed in several resources on variational autoencoders (for example the wikipedia article), we use the same parameters theta for the prior, likelihood, posterior, etc distributions. For example ...
Marko's user avatar
  • 31
0 votes
0 answers
340 views

VAEs vs Autoencoders with BatchNorm and Dropout?

It struck me that regular auto-encoders with batch-norm and dropout have quite similar properties to VAEs which made me wonder whether VAEs where really much better than this simpler alternative. Let ...
profPlum's user avatar
  • 360
2 votes
1 answer
985 views

Could it make any sense to choose a larger dimension for the latent space of the VAE with respect to the original input?

Could it make any sense to choose a larger dimension for the latent space of the VAE with respect to the original input? For example, we may want to learn how to reconstruct a relatively low-...
James Arten's user avatar
0 votes
0 answers
117 views

In CVAE's objective function, why do both terms condition on $\textbf{c}$?

I don't quite understand why, in Conditional Variational Autoencoder (CVAE), we concatenate a conditioning vector two times, at encoder and decoder respectively. After we concatenate it once at the ...
James Arten's user avatar
3 votes
0 answers
266 views

how can a VAE learn to generate a style for Neural Style Transfer?

I have come across this research paper where a Variational Autoencoder is used to map multiple styles from reference images to a linear latent space and then transfer the style to another image like ...
slaifan's user avatar
  • 31
0 votes
0 answers
204 views

How to colorize images with Variational Autoencoder?

CONTEXT I'm trying to colorize images with Variational Autoencoder. Input is 256x256 gray image. Output is 256x256x2 as I convert image to a LAB color space and then put gray channel as input and ...
MASTER OF CODE's user avatar
8 votes
3 answers
1k views

How does backprop work through the random sampling layer in a variational autoencoder?

Implementations of variational autoencoders that I've looked at all include a sampling layer as the last layer of the encoder block. The encoder learns to generate a mean and standard deviation for ...
Luke Wolcott's user avatar
1 vote
0 answers
57 views

Why do we use $q_{\phi}(z \mid x^{(i)})$ in the objective function of amortized variational inference, while sometimes we use $q(z)$?

In page 21 here, it states: General Idea of Amortization: if same inference problem needs to be solved many times, can we parameterize a neural network to solve it? Our case: for all $x^{(i)}$ we ...
a12345's user avatar
  • 213
0 votes
1 answer
213 views

For the VAE, should the input, output and latent variable code be random variables?

For a variational autoencoder, we have input $x$ (assume 1 data point for now, like an image), a latent code sampled from the decoder, $z$, and an output $\hat{x}$. If I were to draw a diagram for the ...
a12345's user avatar
  • 213