Last call to make your voice heard! Our 2022 Developer Survey closes in less than a week. Take survey.

Questions tagged [variational-autoencoder]

For questions related to variational auto-encoders (VAEs). The first VAE was proposed in "Auto-Encoding Variational Bayes" (2013) by Diederik P. Kingma and Max Welling. There are several other VAEs, for example, the conditional VAE.

Filter by
Sorted by
Tagged with
1 vote
0 answers
5 views

Why do we use the same parameters for the joint, marginal and conditional distributions in VAEs?

I've noticed in several resources on variational autoencoders (for example the wikipedia article), we use the same parameters theta for the prior, likelihood, posterior, etc distributions. For example ...
user avatar
  • 11
-1 votes
0 answers
32 views

How is equation 2.2 in the paper "An Introduction to Variational Autoencoders" derived?

I have question about a formula in the paper An Introduction to Variational Autoencoders. In page 9, formula (1.6), I totally agree with it since it is famous formula in the Prof. Koller's book "...
user avatar
  • 1
0 votes
0 answers
23 views

How to generate data lying in the union of different hyperplanes using a VAE

I know that a way to possibly encode prior knowledge into neural networks training is by using differentiable optimization layers (paper). I'm in the following situation, and I'm wondering if it could ...
user avatar
0 votes
0 answers
56 views

Training Variational Autoencoder (VAE) on custom dataset

I am training a VAE on a custom dataset for anomaly detection. The data consists of around 500 images of empty white boxes (at different positions) such as below: Original Image of empty box I am ...
user avatar
0 votes
0 answers
24 views

VAEs vs Autoencoders with BatchNorm and Dropout?

It struck me that regular auto-encoders with batch-norm and dropout have quite similar properties to VAEs which made me wonder whether VAEs where really much better than this simpler alternative. Let ...
user avatar
  • 266
1 vote
1 answer
61 views

Could it make any sense to choose a larger dimension for the latent space of the VAE with respect to the original input?

Could it make any sense to choose a larger dimension for the latent space of the VAE with respect to the original input? For example, we may want to learn how to reconstruct a relatively low-...
user avatar
0 votes
0 answers
19 views

Adding an attention mechanism to variational autoencoder

I'm wondering if it could make sense trying to incorporate an attention mechanism into a variational autoencoder and eventually how to do that. Would it make sense to apply a first layer of processing ...
user avatar
0 votes
0 answers
48 views

In CVAE's objective function, why do both terms condition on $\textbf{c}$?

I don't quite understand why, in Conditional Variational Autoencoder (CVAE), we concatenate a conditioning vector two times, at encoder and decoder respectively. After we concatenate it once at the ...
user avatar
0 votes
0 answers
17 views

Variational AutoEncoders | Is Latent space an Embedding space? [duplicate]

I am learning about Variational Autoencoders and it is mentioned that the objective of an encoder is to produce a latent space, "encoding vector". Question: Is latent space just an "...
user avatar
  • 135
0 votes
0 answers
12 views

Incorporate specific constraints while training a (Conditional) variational autoencoder

I'm wondering how could I incorporate specific constraints during the training phase of a deep learning model. In particular, I work for a materials-science related project where I feed to my models ...
user avatar
3 votes
0 answers
43 views

how can a VAE learn to generate a style for Neural Style Transfer?

I have come across this research paper where a Variational Autoencoder is used to map multiple styles from reference images to a linear latent space and then transfer the style to another image like ...
user avatar
  • 31
0 votes
0 answers
31 views

How to colorize images with Variational Autoencoder?

CONTEXT I'm trying to colorize images with Variational Autoencoder. Input is 256x256 gray image. Output is 256x256x2 as I convert image to a LAB color space and then put gray channel as input and ...
user avatar
0 votes
0 answers
10 views

Can Dynamical Variational Auto-encoders be trained on and used to generate static 2D images?

Is it possible to train dynamical variational autoencoders, such as Kalman Variational Autoencoders (KVAE), Recurrent Variational Autoencoders (RVAE), or Disentangled Sequential Autoencoders (DSAE) on ...
user avatar
6 votes
3 answers
186 views

How does backprop work through the random sampling layer in a variational autoencoder?

Implementations of variational autoencoders that I've looked at all include a sampling layer as the last layer of the encoder block. The encoder learns to generate a mean and standard deviation for ...
user avatar
1 vote
0 answers
32 views

Why do we use $q_{\phi}(z \mid x^{(i)})$ in the objective function of amortized variational inference, while sometimes we use $q(z)$?

In page 21 here, it states: General Idea of Amortization: if same inference problem needs to be solved many times, can we parameterize a neural network to solve it? Our case: for all $x^{(i)}$ we ...
user avatar
  • 203
0 votes
1 answer
52 views

For the VAE, should the input, output and latent variable code be random variables?

For a variational autoencoder, we have input $x$ (assume 1 data point for now, like an image), a latent code sampled from the decoder, $z$, and an output $\hat{x}$. If I were to draw a diagram for the ...
user avatar
  • 203
1 vote
1 answer
49 views

Do we use two distinct layers to compute the mean and variance of a Gaussian encoder/decoder in the VAE?

I am looking at appendix C of the VAE paper: It says: C.1 Bernoulli MLP as decoder In this case let $p_{\boldsymbol{\theta}}(\mathbf{x} \mid \mathbf{z})$ be a multivariate Bernoulli whose ...
user avatar
  • 203
2 votes
3 answers
124 views

Are the authors of the VAE paper writing the PDFs as a function of the random variables?

Usually, I see the conventions: discrete random variable is denoted as $X$, the pmf is written as $P(X=x)$ or $p(X=x)$ or $p_{X}(x)$ or $p(x)$, where $x$ is an instance of $X$ a continuous random ...
user avatar
  • 203
1 vote
0 answers
65 views

How to estimate conditional density using neural network?

Conditional Variational Autoencoders (CVAE) and Mixture Density Networks (MDN) are supposed to address this issue. However, these models provide the distribution parameters, e.g., mean and standard ...
user avatar
0 votes
0 answers
25 views

Adversarial Autoencoder is not working and not learning properly

I am trying to get an Adversarial AutoEncoder going using keras Fit method on a keras.model class but for some reason it is not working. Keep in mind that I tried updating encoder and decoder at the ...
user avatar
  • 1
1 vote
1 answer
148 views

How is the VAE related to the Autoencoding Variational Bayes (AEVB) algorithm?

I am familiar with the variational autoencoder, but not totally clear on what exactly the AEVB is. In the original VAE paper (by Kingma and Welling), he uses both the terms variational autoencoder and ...
user avatar
  • 203
1 vote
1 answer
188 views

How does the VAE learn a joint distribution?

I found the following paragraph from An Introduction to Variational Autoencoders sounds relevant, but I am not fully understanding it. A VAE learns stochastic mappings between an observed $\mathbf{x}$...
user avatar
  • 203
1 vote
0 answers
176 views

What is the most suitable measure of the distance between two VAE's latent spaces?

The problem I'm trying to solve is as follows. I have two separate domains, where inputs do not have the same dimensions. However, I want to create a common feature space between both domains using ...
user avatar
3 votes
1 answer
74 views

What are the roles of the prior $\mathrm{p}(\mathbf{z})$ in a VAE?

I know the encoder is variational posterior $q_{\phi}(\mathbf{z} \mid \mathbf{x})$. I also know that the decoder represents the likelihood: $p_{\theta}(\mathbf{x} \mid \mathbf{z})$. My question is ...
user avatar
  • 203
3 votes
1 answer
105 views

How to determine the quality of synthetic data?

I'm working on a VAE model to produce synthetic data of X-Ray diffraction spectrums. I try to figure out how I can measure the quality of the spectrums. The goal would be to produce synthetic data ...
user avatar
  • 33
0 votes
0 answers
47 views

variational autoencoder - decoder output for images

Following the standard setup/notation for a VAE, let $z$ denote the latent variables, $q$ as the encoder, $p$ as the decoder, and $x$ as the label. Let the objective be to maximize the ELBO, where a ...
user avatar
  • 746
0 votes
1 answer
38 views

Are some low dimensional distributions known to be hard to model with VAEs?

I am trying to implement a toy VAE project. My goal is to use a VAE to model the moon dataset from scikit-learn, with an extra constant (but noisy) z-dimension. To this end I use an approximate ...
user avatar
  • 21
0 votes
0 answers
22 views

Tensorflow Probability Implementation of Automatic Differentiation Variational Inference with Mixtures

In this paper, the authors suggest using the following loss instead of the traditional ELBO in order to train what basically is a Variational Autoencoder with a Gaussian Mixture Model instead of a ...
user avatar
  • 1
1 vote
1 answer
30 views

Why is the prior on the latent variable standard gaussian in VAE?

While training a standard VAE, we assume that the prior on the latent variable Z is the standard gaussian and we use KL divergence to push the posterior as close as possible to the standard gaussian. ...
user avatar
0 votes
0 answers
31 views

Types of decoder parametrizations in VAE for continuous data

I'm wondering what are the different choices of parametrizations available for the decoder in a variational autoencoder. If the data is discrete, you can just output probabilities for each class, so ...
user avatar
  • 746
1 vote
2 answers
184 views

Are mean and standard deviation in variational autoencoders unique?

In general, if I have a collection of data then mean(Expectation) and standard deviation are calculated as follows $$\text{mean } = \mu = \mathbb{E}[X] = \sum\limits_{i = 1}^n p_ix_i $$ $$\text{...
user avatar
  • 3,059
0 votes
0 answers
123 views

Can I use the VAE for dimensionality reduction?

I'm doing a project that uses a clustering algorithm for the facial expression classification task. So, I use the output of the encoder in the VAE autoencoder for dimensionality reduction. However, I ...
user avatar
0 votes
1 answer
202 views

Is there a performace benefits using VAE-GAN instead of just GAN?

I have read that when using VAE-GANs, first what happens is the VAE's encoder encodes some image to another encoded image, which from GAN's point of view is considered a noise, and then the GAN part ...
user avatar
0 votes
0 answers
42 views

Problems while transforming a 2D Variational Autoencoder into a 1D Version

I am trying to addapt the Keras variational autoencoder (VAE) here from a 2-D input/output (matrix of a picture) to a 1-D input/output (just a vector). I thought this would be a fearly easy task, but ...
user avatar
0 votes
0 answers
31 views

Weird KL divergence behaviour

I'm training a complex model for motion prediction using a VAE, however the KL divergence has a very strange behavior. A scheleton of the network is the following: At the end my network compute the ...
user avatar
  • 149
1 vote
0 answers
40 views

Doubts on how this Dimensions are handled (Tensorflow) and how mean and standard deviation are extracted [closed]

I'm following the introductory MIT Deep Learning course on Youtube and i've been stuck for a day now on this piece of code: ...
user avatar
3 votes
1 answer
98 views

What does the approximate posterior on latent variables, $q_\phi(z|x)$, tend to when optimising VAE's

The ELBO objective is described as follows $$ ELBO(\phi,\theta) = E_{q_\phi(z|x)}[log p_\theta (x|z)] - KL[q_\phi (z|x)||p(z)] $$ This form of ELBO includes a regularisation term in the form of the ...
user avatar
1 vote
1 answer
40 views

Are there architectures to generate pictures from four labels? (VAEs, GANs)

I want to try something with image creation via NNs. I have come across Variational Autoencoders and Generative Adversarial Networks as possible solutions but have only found image creatinon with ...
user avatar
0 votes
2 answers
619 views

In variational autoencoders, why do people use MSE for the loss?

In VAEs, we try to maximize the ELBO $\mathbb(E_q log\ p(x|z) + D_{KL}(q(z|x), p(z))$), but I see that many implement the first term as MSE of the image and it's reconstruction. Is this mathematically ...
user avatar
  • 129
1 vote
2 answers
94 views

In variational autoencoders, what does p(x|z) mean?

If $x \sim \mathcal{N}(\mu,\,\sigma^{2})$, then it is a continuous variable, and therefore $P(x) = 0$ for any x. One can only consider things like $P(x<X)$ to get a probability greater than 0. So ...
user avatar
  • 129
1 vote
1 answer
98 views

In VQ-VAE code what does this line of code signify?

The VQ-VAE implimentation:https://colab.research.google.com/github/zalandoresearch/pytorch-vq-vae/blob/master/vq-vae.ipynb ...
user avatar
  • 31
1 vote
0 answers
208 views

Why does the VAE using a KL-divergence with a non-standard mean does not produce good images?

I know I can make a VAE do generation with a mean of 0 and std-dev of 1. I tested it with the following loss function: ...
user avatar
  • 43
6 votes
2 answers
543 views

Why is "-0.5 * torch.sum(1 + sigma - mu.pow(2) - sigma.exp())" in Pytorch equivalent to the KL?

I found this code for the loss function of a VAE: -0.5 * torch.sum(1 + sigma - mu.pow(2) - sigma.exp()) From this link: https://debuggercafe.com/getting-started-...
user avatar
3 votes
1 answer
110 views

How do you calculate KL divergence on a three-dimensional space for a Variational Autoencoder?

I'm trying to implement a variational auto-encoder (as seen in Section 3.1 here: https://arxiv.org/pdf/2004.06271.pdf). It differs from a traditional VAE because it encodes its input images to three-...
user avatar
1 vote
0 answers
51 views

variational auto encoder loss goes down but does not reconstruct input. out of debugging ideas

My variational autoencoder seems to work for MNIST, but fails on slightly "harder" data. By "fails" I mean there are at least two apparent problems: Very poor reconstruction, for ...
user avatar
  • 669
1 vote
0 answers
158 views

VAE giving near zero output when latent space dimension is large

I'm training a VAE to reconstruct some input (channels picked up by some MIMO BS for context) and I ran an experiment on the training set to see how the performance improves with the latent space ...
user avatar
8 votes
1 answer
4k views

What are the fundamental differences between VAE and GAN for image generation?

Starting from my own understanding, and scoped to the purpose of image generation, I'm well aware of the major architectural differences: A GAN's generator samples from a relatively low dimensional ...
user avatar
1 vote
0 answers
137 views

Hairstyle Virtual Try On

I want to help people with cancer who are under chemotherapy, and generally people who have lost their hair to Virtually Try-On Toupees/Wigs on their head. VTO must support both the frontal and side ...
user avatar
  • 111
2 votes
1 answer
50 views

What do they mean by "contradictory loss"?

In page 4 of the paper https://arxiv.org/pdf/2009.07047v1.pdf, it says the encoder $E_{R,X}$ of $VAE_1$ tries to fool the discriminator with a contradictory loss to ensure that $R$ and $X$ are mapped ...
user avatar
  • 21
2 votes
0 answers
30 views

Compressing Parameters of an Response System

I have an input-output system, which is fully determined by 256 parameters, of which I know a significant amount are of less importance to the input-output pattern. The data I have is some (64k in ...
user avatar
  • 121