Skip to main content

Questions tagged [variational-inference]

For questions related to variational inference (VI), an optimization-based approach to the inference problem (i.e. the computation of the posterior given the prior, likelihood, and marginal). VI is used, for example, in the context of auto-encoders (VAEs) and Bayesian neural networks (BNNs).

Filter by
Sorted by
Tagged with
0 votes
0 answers
21 views

Is this right logic flow of Variational Autoencoder?

Object Function $$ maximize = \Pi_d^D P(X_d) = \Pi_d^D \sum_i^N P(X_d,Z_i) $$ D : # of data N : # of latent variable state $$ NLL = - \sum_d^D \log \sum_i^N P(X_d,Z_i) = - \sum_d^D \log \sum_i^N P(X_d,...
Shin Joong Hyun's user avatar
0 votes
0 answers
26 views

Does the log determinant loss term mean that there is no need to use weight decay?

In training affine coupling layers in a normalizing flow, does the log determinant loss term mean that there is no need to use weight decay? Since the loss term constrains the size of the ...
Richie Bendall's user avatar
2 votes
2 answers
162 views

What is the meaning of log p(x) in VAE math and why is it constant

I was reading the article on medium, where the author cites this equation for Variational Inference: \begin{align*} \text{KL}(q(z|x^{(i)})||p(z|x^{(i)})) &= \int_z q(z|x^{(i)})\text{log}\frac{q(z|...
Kiran Manicka's user avatar
1 vote
1 answer
119 views

How does using the ELBO in VAEs make the problem tractable?

I'm studying Variational Autoencoders and a lot of the literature says that the posterior is intractable because the marginal distribution p(x) is intractable since the space of z is so large we ...
Kiran Manicka's user avatar
0 votes
0 answers
34 views

Variational Autoencoders - Can We Learn Directly From Marginal With a Pretrained Decoder?

So, with VAE we use ELBO instead of directly maximizing the marginal likelihood, because the marginal likelihood is intractable. As far as I understand it, this is the case for two reasons: $$p(x) = \...
BurgerMan's user avatar
0 votes
0 answers
17 views

Trouble understanding Variational Inference objective

I was reading the Meta Temporal Point Processes paper and was having trouble understading the training objective presented. The authors state that it is the ELBO used in Variational Inference ...
YEp d's user avatar
  • 1
1 vote
1 answer
76 views

If we know the joint distribution, can we simply derive the evidence from it?

I'm struggling to understand one specific part of the formalism of the free energy principle. My understanding is that the free energy principle can be derived from considering statistical dynamics of ...
Gustavo's user avatar
  • 11
1 vote
2 answers
358 views

Why optimise log p(x) rather than log p(x|z) in a Variational AutoEncoder?

Background The loss function in a Variational AutoEncoder is the Evidence Lower Bound (ELBO): $\mathbb{E}_q[log$ $p(x|z)] - KL[q(z)||p(z)]$ And has this inequality: $log$ $p(x) \ge \mathbb{E}_q[log$ $...
Titus Buckworth's user avatar
0 votes
1 answer
72 views

Why isn't the evidence $p(x) = 1$ if it's an observed variable?

Every explanation of variational inference starts with the same basic premise: given an observed variable $x$, and a latent variable $z$, $$ p(z|x)=\frac{p(x,z)}{p(x)} $$ and then proceeds to expand $...
Abrrval's user avatar
2 votes
1 answer
485 views

Why don't we also need to approximate $p(x \mid z)$ in the VAE?

In the VAE, we approximate the probability distribution $p(z \mid x)$, where $z$ is the latent vector and $x$ is our data. The reason is that $p(z \mid x)$ becomes impossible to calculate for ...
Nervous Hero's user avatar
1 vote
1 answer
85 views

Why do we use $q_{\phi}(z \mid x^{(i)})$ in the objective function of amortized variational inference, while sometimes we use $q(z)$?

In page 21 here, it states: General Idea of Amortization: if same inference problem needs to be solved many times, can we parameterize a neural network to solve it? Our case: for all $x^{(i)}$ we ...
a12345's user avatar
  • 233
1 vote
1 answer
299 views

Do we use two distinct layers to compute the mean and variance of a Gaussian encoder/decoder in the VAE?

I am looking at appendix C of the VAE paper: It says: C.1 Bernoulli MLP as decoder In this case let $p_{\boldsymbol{\theta}}(\mathbf{x} \mid \mathbf{z})$ be a multivariate Bernoulli whose ...
a12345's user avatar
  • 233
4 votes
1 answer
1k views

How does the VAE learn a joint distribution?

I found the following paragraph from An Introduction to Variational Autoencoders sounds relevant, but I am not fully understanding it. A VAE learns stochastic mappings between an observed $\mathbf{x}$...
a12345's user avatar
  • 233
0 votes
0 answers
53 views

Tensorflow Probability Implementation of Automatic Differentiation Variational Inference with Mixtures

In this paper, the authors suggest using the following loss instead of the traditional ELBO in order to train what basically is a Variational Autoencoder with a Gaussian Mixture Model instead of a ...
jonas's user avatar
  • 1
3 votes
1 answer
277 views

What does the approximate posterior on latent variables, $q_\phi(z|x)$, tend to when optimising VAE's

The ELBO objective is described as follows $$ ELBO(\phi,\theta) = E_{q_\phi(z|x)}[log p_\theta (x|z)] - KL[q_\phi (z|x)||p(z)] $$ This form of ELBO includes a regularisation term in the form of the ...
quest ions's user avatar
5 votes
1 answer
707 views

What is the intuition behind variational inference for Bayesian neural networks?

I'm trying to understand the concept of Variational Inference for BNNs. My source is this work. The aim is to minimize the divergence between the approx. distribution and the true posterior $$\text{KL}...
f_3464gh's user avatar