All Questions
Tagged with vae or variational-autoencoder
140 questions
1
vote
0
answers
22
views
KL annealing for a VAE does not work, what now?
I am trying to train a variational auto-encoder where x ≈ f_VAE(x) = x_hat. In my real problem, I have 100-400 dimensional data that I would like to compress to ...
- 131
-1
votes
0
answers
12
views
Relation between word embeddings to a more general encoding (such as in VAE, contrastive learning, and intermediate layers)
Contrastive learning finds a representation in an m-dimensional space, where similar examples will be close to each other and dissimilar examples will lie far away. Similarly, word embeddings are a ...
- 11
1
vote
1
answer
35
views
Does anyone use Statistical Energy to monitor generative AI training?
Statistical Energy (Szekely & Rizzo, 2013 or Aslan & Zech, 2005) can be used as a statistical test of whether two distributions are the same or different. It works particularly well on high ...
- 111
0
votes
0
answers
15
views
Which generative model architecture (and loss function) should I use to train on a dataset of 3D arrays of embedded tokens?
I've curated a dataset of player-made Minecraft builds. Each unique Minecraft block is tokenized and treated as a unique "word" like in NLP. I've trained a Skip-Gram model on the dataset (...
- 1
0
votes
0
answers
22
views
VAE to compare two datasets
I have two cell-by-gene matrices, each representing gene counts for cells in sample 1 and sample 2. I'm interested in identifying common gene expression patterns across both samples. These patterns ...
- 111
2
votes
1
answer
69
views
Question regarding the ELBO decomposition proposed by Hoffman&Johnson
recently I'm trying to read a paper by Hoffman and Johnson discussing an alternative decomposition of ELBO in variational autoencoders. In formula (9) and (10) of their original paper, they proposed ...
- 23
1
vote
1
answer
47
views
MLP Gaussian Decoder in VAE
My question concerns the paper arxiv.org/pdf/1312.6114. I want to know why they proposed to use MLP Gaussian decoder with parameters given by the MLP transformation of the z variable as the likelihood ...
- 133
2
votes
1
answer
179
views
Deriving ELBO for Diffusion Models
I am trying to read through the proof of ELBO for diffusion models on pg. 8 of this paper. However, I do not see how the author arrived at Eqn (45) from Eqn (44). Specifically, I do not know how they ...
1
vote
0
answers
29
views
Any tutorials/courses to learn variational autoencoders on tabular data?
I aim to use variational autoencoders (VAE) to find interpretable latent spaces for genetic data. So, I need to understand how they work, what activation function to use, etc. But all tutorials and ...
- 111
2
votes
0
answers
83
views
VAE suffers from posterior collapse under all hyper parameters [closed]
I am trying to find a low-dimensional latent space representation for a bunch of simulated data. No matter what VAE architecture I try and no matter how I tweak it, the output of the VAE is always the ...
- 131
2
votes
1
answer
93
views
Derivation of the consistency term in the DDPM Evidence Lower Bound (ELBO) [closed]
I have been studying diffusion models from this tutorial: https://arxiv.org/abs/2403.18103 and trying to derive all results as I read it. Although this tutorial is very comprehensive, it skips many of ...
- 31
0
votes
0
answers
104
views
How to solve the exploding gradient problem in VAE training?
I was trying to implement VAE on the CelebA dataset inspired by the Tensorflow implementation of MNIST. I have tried varying batch size but there seems to be no effect from that. The image formed is ...
0
votes
0
answers
25
views
In VAE, why not maximize D[q(z)||p(z|x)] instead？
In VAE, the goal is to maximize $\log p(x)$, where $x$ is the data and $p(\cdot)$ is a parameterized distribution.
For any distribution $q(z)$, the following identity holds,
$$
\log p(x)=D[q(z)||p(z|x)...
- 1
0
votes
1
answer
112
views
Why do we add noise when training VAEs, instead of just insisting each training batch is Normally distributed at hidden layer?
I had come to think of VAEs as having 2 loss terms - A reconstruction loss that tries to ensure that the input matches the output (just as a regular Auto Encoder), and a KL-Divergence Loss term that ...
- 1
1
vote
1
answer
218
views
Trying to understand some derivation in the paper: Deep Unsupervised Learning using Nonequilibrium Thermodynamics
I have recently been learning about diffusion models and trying to derive all the results in the paper by Sohl-Dickstein, et. al, "Deep Unsupervised Learning using Nonequilibrium Thermodynamics&...
- 31
0
votes
1
answer
79
views
How to remove random noise from an image (denoising)?
When adding noise to an image, for instance,
is the noise added evenly random (equally likely values within some range), or random but following some distribution (like the normal distribution)?
Then,...
- 157
3
votes
1
answer
360
views
How to perform latent space Interpolation between two images?
I have a variational convolutional autoencoder that has trained on 2 images and outputs a linear interpolation (inserted at the bottleneck stage) between those 2 input images.
However, the result ...
- 157
3
votes
0
answers
66
views
How random should an untrained generative AI output really be?
I am developing a particular implementation of VAE, and, how usually one does while implementing any architecture, I passed a random input to the model to test if everything worked fine (e.g. check ...
- 31
4
votes
3
answers
221
views
In the VAE, why is $z \sim \mathcal{N}(\mu, \sigma^2)$ equivalent to $z = \mu + \sigma \odot \epsilon$?
In the reparameterization trick of a Variational Autoencoder (VAE), instead of sampling noise $z$ from $z \sim \mathcal{N}(\mu, \sigma^2)$, we can use a different method: $z = \mu + \sigma \odot \...
user77925
0
votes
0
answers
26
views
Is there a way to mix the Importance weighted VAE with the Beta-VAE?
Is there a way to mix the Importance weighted VAE with the Beta-VAE?
It seems that we cannot separate out the KL-term from the IWAE-ELBO, hence we cannot multiply it with Beta. On the other hand, it ...
- 11
2
votes
2
answers
288
views
What is the meaning of log p(x) in VAE math and why is it constant
I was reading the article on medium, where the author cites this equation for Variational Inference:
\begin{align*}
\text{KL}(q(z|x^{(i)})||p(z|x^{(i)})) &= \int_z q(z|x^{(i)})\text{log}\frac{q(z|...
- 103
1
vote
1
answer
198
views
How does using the ELBO in VAEs make the problem tractable?
I'm studying Variational Autoencoders and a lot of the literature says that the posterior is intractable because the marginal distribution p(x) is intractable since the space of z is so large we ...
- 103
0
votes
1
answer
125
views
Are there cases where Variational Auto-Encoders (VAE's) are preferred to other techniques?
The best reason I have seen for using variational autoencoders is when dealing with sparse data. The Gaussian noise "splats" out the input distribution (see this StackExchange answer).
...
- 166
0
votes
0
answers
60
views
What best practices for VAE do you know?
The data is binary voxel data of shape (60, 36, 60). I want to compress such data into ...
0
votes
1
answer
832
views
Why does the latent space in Stable Diffusion have a shape of 64x64x3?
Since the encoding is performed by a Variational Autoencoder, the VAE encoder must output some mean and log variance that we can ...
0
votes
0
answers
24
views
How to apply Latent Diffusion for 3D Binary Voxel Data?
Suppose we have a voxel of shape (60, 36, 60) with values 0 or 1 (1-occupied, 0-empty). What is the possible architecture of latent diffusion?
0
votes
0
answers
39
views
Which main steps should I consider in order to successfully use a VAE for Anomaly Detection?
I am thinking about using the variational autoencoder model for anomaly detection . I have an Android Logs dataset. As the logs generated are a representative of time series type of data I thought ...
0
votes
0
answers
83
views
VAE ( variational autoencoder) for timeseries anomaly detection ,
I am implementing VAE based anomaly detection for multivariate timeseries using keras,
I have ELBO (Evidence lower bound) which is combination of
$$-\ D_{KL}\left({\ q}_\varphi\left(z\middle| x^i\...
- 1
1
vote
1
answer
99
views
Variational Lower Bound in VAE for Gaussian latent prior
From Bishop's recent book on Deep Learning, it says the ELBO for Gaussian latent prior can be approximated by
$\frac{1}{L}\sum_{l=1}^L \ln p(x_n|z_n^l,w) + KL(q(z_n|x_n,\phi)||p(z_n))$
where $n$ are ...
- 133
1
vote
1
answer
190
views
What is an information bottleneck in the context of ELBO and Hierarchical VAEs?
These slides (slide number 26) mention that the ELBO enforces an information bottleneck at the latent variables z which make it prone to bad local minima.
Can you please explain what they mean by that?...
1
vote
3
answers
191
views
What is the difference between q and p in Statistical Notation(used in VAE)?
I'm looking at general visuals of Variational Autoencoders and I'm seeing that the encoder is typically expressed as q(z|x) with phi as a subscript while the decoder is p(x|z) with theta as a ...
- 103
1
vote
1
answer
91
views
Is there a way to reward my Variational-Auto encoder for using less colors while still letting it make creative decisions?
So recently I have been trying to program a Tensorflow and Keras based model that can animate pixel art characters. I use a Variation Auto Encoder with Convolutions, Dense Layers and Upsampling 2d ...
- 13
1
vote
0
answers
93
views
Pointers to (deep) latent variable models that admit analytical approximations
I am aware that there is a plethora of deep generative models out there (e.g. variational autoencoders (VAE), GANs) that can model high-dimensional data as the images of latent variables under a non-...
- 111
0
votes
0
answers
41
views
VAE for Motion Sequence Generation - Convergence Issue with Scheduled Sampling
I have implemented a Variational Autoencoder (VAE) in PyTorch for motion sequence generation using human pose data (joint angles and angular velocities in radians) from the CMU dataset. The VAE ...
- 1
1
vote
0
answers
29
views
Toy dataset: Radial VAE
I'm evaluating disentanglement in toy datasets seeing as we have such little understanding of the phenomena. I'm using various tools from differential geometry. Now I want to train a VAE on the ...
- 111
1
vote
1
answer
571
views
Variational Autoencoder (VAE) Multiclass Interpolation
I'm working on a variational autoencoder (VAE) with 20 different classes in my training data. I've successfully trained the VAE and can sample from the latent space to generate data points. However, I ...
2
votes
1
answer
988
views
What exactly is meant by variational distribution?
What specifically does the term "variational distribution" refer to?
The encoder of Variational autoencoder?
Forward process of denoising diffusion probabilistic models?
Images or latent ...
0
votes
1
answer
126
views
Confusion over taking gradients in Variational Autoencoders (VAE)
I am confused as to when to hold certain parameters constant in a VAE. I will explain with a concrete example.
We can write $\operatorname{ELBO}(\phi, \theta) = \mathbb{E}_{q_{\phi}(z)}\left[\log
\...
2
votes
1
answer
143
views
Why can Variational Autoencoders (VAEs) approximate arbitrary distributions?
I am trying to reason to myself why is it that VAEs can approximate arbitrary probability distributions even though 𝑞𝜙(𝑧|𝑥)
and 𝑝𝜃(𝑥|𝑧)
are Gaussian.
I understand that the parameters are ...
0
votes
2
answers
163
views
How to expand reconstruction error to mean squared error in Variational AutoEncoder? [closed]
How to expand reconstruction error to mean squared error when it is $\mathbb{E}_{z\sim q_{\phi}(z|x)}[\log p_\theta(x|z)]$?
[reconstruction error]
$\mathbb{E}_{z\sim q_{\phi}(z|x)}[\log p_\theta(x|z)]$...
1
vote
1
answer
192
views
Comparison of the two alternative forms for the KL divergence [closed]
On page 468 of 'Pattern Recognition and Machine Learning', what does 'the same variables given by the product of two independent univariate Gaussian distributions' mean?
The PDF says,
The green ...
3
votes
0
answers
178
views
Why is $p(x)=\int p(x,z) dz$ intractable for continuous $z$ in VAE?
In VAE we use the importance sampling trick to use $q_\phi(z|x)$ to help maximize $\log p_\theta(x)=\log \int p_\theta(x,z)dz\ge \int q_\phi(z|x)\log \frac{p_\theta(x,z)}{q_\phi(z|x)} dz$. Meanwhile, ...
- 31
1
vote
2
answers
65
views
How to apply backpropagation when one layer of the network is a call-only function (no gradient)?
I worked with Feed Forward Neural Network and VAE and understood backpropagation algorithm. Now I build a VAE network, one layer of it is a very complex vector-to-vector function $f(x)$ (a general '...
- 111
0
votes
1
answer
234
views
Any suggestion on what I should try to get this cVAE model working for random generation of new molecules?
I am using a cVAE model to generate new structures of molecules. After successfully training the VAE model I am able to get proper reconstructions of the training set. While I am also able to generate ...
- 135
0
votes
1
answer
33
views
How to convert my test data in the same dimensionality as my train data
I have trained a VAE with jpg images. My latent space dimension has 768 features and when plotting the latent space it looks like this:
However, when I use the scikit learn tool LDA (Linear ...
- 3
1
vote
2
answers
505
views
Why optimise log p(x) rather than log p(x|z) in a Variational AutoEncoder?
Background
The loss function in a Variational AutoEncoder is the Evidence Lower Bound (ELBO):
$\mathbb{E}_q[log$ $p(x|z)] - KL[q(z)||p(z)]$
And has this inequality:
$log$ $p(x) \ge \mathbb{E}_q[log$ $...
- 111
0
votes
1
answer
59
views
Why is the variational lower bound is easier to compute than the original marginal distribution?
Why is the ELBO of $p(x)=\int p(x|z)p(z)\mathrm{d}z$ easier to compute/estimate than the expression itself? Can we compute this quantity itself through sampling in the same way? I understanding that ...
- 101
0
votes
0
answers
230
views
Using a VQ-VAE encoder's output to condition another model
I've trained a VQ-VAE on images that I want to use to condition another model (latent diffusion). The shape of my encoder's output is (4, 256, 56, 56) and I'm getting it using:
...
- 101
-1
votes
1
answer
243
views
Combination of VAE with GAN
I am going to implement a lecture which it aims to generate new images. It uses a variational autoencoder to produce latent vector and then feed it to a gan network as input. My question is, in ...
0
votes
1
answer
515
views
How to generate new data using VAE?
I have built the following function which takes as input some data and runs a VAE on them:
...
- 101