8
votes
Accepted
What is the intuition behind variational inference for Bayesian neural networks?
Your description of what is going on is more or less correct, although I am not completely sure that you have really understood it, given your last question.
So, let me enumerate the steps.
The ...
- 39.7k
3
votes
Accepted
Why don't we also need to approximate $p(x \mid z)$ in the VAE?
What I can guess here is that, in VAEs, we assume $p(z)$ (prior), so we are able to calculate $p(x \mid z)$, but for $p(x)$ we can't assume its distribution? Is it right?
You could assume $p(x)$ is ...
- 39.7k
2
votes
Accepted
How does the VAE learn a joint distribution?
The VAE models the following directed graphical model (figure 1 from the original VAE paper)
So, you have 2 sets of parameters, $\boldsymbol{\phi}$ and $\boldsymbol{\theta}$, and 2 random variables, $...
- 39.7k
1
vote
If we know the joint distribution, can we simply derive the evidence from it?
If the full joint distribution $p(o,x)=p(o|x)p(x)$ (usually assumed to be the product of two Gaussians for free energy principle of predictive coding, and in many practical situations you can only ...
- 960
1
vote
Why optimise log p(x) rather than log p(x|z) in a Variational AutoEncoder?
I thought $p(x)$ was just the distribution of $x$, which are the input variables we have observed. So how can we maximize $p(x)$, if it is the actual distribution of data in the real world?
I think ...
- 2,426
1
vote
Why optimise log p(x) rather than log p(x|z) in a Variational AutoEncoder?
TLDR: We're doing a maximum likelihood fit of our model. The VAE sets this up in a way that doesn't require evaluating the model likelihood, but instead expresses a lower bound in terms of ...
- 56
1
vote
Accepted
Do we use two distinct layers to compute the mean and variance of a Gaussian encoder/decoder in the VAE?
Yes, in the case of the Gaussian, you have two distinct layers (so weights and biases), one for the mean and the other for the variance, as the equations are telling us.
The mean is calculated with ...
- 39.7k
1
vote
What does the approximate posterior on latent variables, $q_\phi(z|x)$, tend to when optimising VAE's
Practically, when optimizing VAE, you assume that prior $p(z)\sim N(0,1)$; i.e. the unit Gaussian distribution. However, in testime you sample z from $p(z|x)$; the encoder model. Why is that?
Let's go ...
- 919
Only top scored, non community-wiki answers of a minimum length are eligible