Skip to main content

Questions tagged [variational-inference]

For questions related to variational inference (VI), an optimization-based approach to the inference problem (i.e. the computation of the posterior given the prior, likelihood, and marginal). VI is used, for example, in the context of auto-encoders (VAEs) and Bayesian neural networks (BNNs).

9 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
2 votes
2 answers
169 views

What is the meaning of log p(x) in VAE math and why is it constant

I was reading the article on medium, where the author cites this equation for Variational Inference: \begin{align*} \text{KL}(q(z|x^{(i)})||p(z|x^{(i)})) &= \int_z q(z|x^{(i)})\text{log}\frac{q(z|...
Kiran Manicka's user avatar
1 vote
1 answer
126 views

How does using the ELBO in VAEs make the problem tractable?

I'm studying Variational Autoencoders and a lot of the literature says that the posterior is intractable because the marginal distribution p(x) is intractable since the space of z is so large we ...
Kiran Manicka's user avatar
1 vote
1 answer
85 views

Why do we use $q_{\phi}(z \mid x^{(i)})$ in the objective function of amortized variational inference, while sometimes we use $q(z)$?

In page 21 here, it states: General Idea of Amortization: if same inference problem needs to be solved many times, can we parameterize a neural network to solve it? Our case: for all $x^{(i)}$ we ...
a12345's user avatar
  • 233
0 votes
0 answers
21 views

Is this right logic flow of Variational Autoencoder?

Object Function $$ maximize = \Pi_d^D P(X_d) = \Pi_d^D \sum_i^N P(X_d,Z_i) $$ D : # of data N : # of latent variable state $$ NLL = - \sum_d^D \log \sum_i^N P(X_d,Z_i) = - \sum_d^D \log \sum_i^N P(X_d,...
Shin Joong Hyun's user avatar
0 votes
0 answers
26 views

Does the log determinant loss term mean that there is no need to use weight decay?

In training affine coupling layers in a normalizing flow, does the log determinant loss term mean that there is no need to use weight decay? Since the loss term constrains the size of the ...
Richie Bendall's user avatar
0 votes
0 answers
37 views

Variational Autoencoders - Can We Learn Directly From Marginal With a Pretrained Decoder?

So, with VAE we use ELBO instead of directly maximizing the marginal likelihood, because the marginal likelihood is intractable. As far as I understand it, this is the case for two reasons: $$p(x) = \...
BurgerMan's user avatar
0 votes
0 answers
17 views

Trouble understanding Variational Inference objective

I was reading the Meta Temporal Point Processes paper and was having trouble understading the training objective presented. The authors state that it is the ELBO used in Variational Inference ...
YEp d's user avatar
  • 1
0 votes
1 answer
72 views

Why isn't the evidence $p(x) = 1$ if it's an observed variable?

Every explanation of variational inference starts with the same basic premise: given an observed variable $x$, and a latent variable $z$, $$ p(z|x)=\frac{p(x,z)}{p(x)} $$ and then proceeds to expand $...
Abrrval's user avatar
0 votes
0 answers
53 views

Tensorflow Probability Implementation of Automatic Differentiation Variational Inference with Mixtures

In this paper, the authors suggest using the following loss instead of the traditional ELBO in order to train what basically is a Variational Autoencoder with a Gaussian Mixture Model instead of a ...
jonas's user avatar
  • 1