Questions tagged [variational-inference]

For questions related to variational inference (VI), an optimization-based approach to the inference problem (i.e. the computation of the posterior given the prior, likelihood, and marginal). VI is used, for example, in the context of auto-encoders (VAEs) and Bayesian neural networks (BNNs).

Filter by
Sorted by
Tagged with
0 votes
1 answer
20 views

Very basic variational inference question

Every explanation of variational inference starts with the same basic premise: given an observed variable $x$, and a latent variable $z$, $$ p(z|x)=\frac{p(x,z)}{p(x)} $$ and then proceeds to expand $...
1 vote
1 answer
86 views

Why don't we also need to approximate $p(x \mid z)$ in the VAE?

In the VAE, we approximate the probability distribution $p(z \mid x)$, where $z$ is the latent vector and $x$ is our data. The reason is that $p(z \mid x)$ becomes impossible to calculate for ...
1 vote
0 answers
40 views

Why do we use $q_{\phi}(z \mid x^{(i)})$ in the objective function of amortized variational inference, while sometimes we use $q(z)$?

In page 21 here, it states: General Idea of Amortization: if same inference problem needs to be solved many times, can we parameterize a neural network to solve it? Our case: for all $x^{(i)}$ we ...
  • 203
1 vote
1 answer
68 views

Do we use two distinct layers to compute the mean and variance of a Gaussian encoder/decoder in the VAE?

I am looking at appendix C of the VAE paper: It says: C.1 Bernoulli MLP as decoder In this case let $p_{\boldsymbol{\theta}}(\mathbf{x} \mid \mathbf{z})$ be a multivariate Bernoulli whose ...
  • 203
0 votes
0 answers
40 views

Variational Inference: Approximate expected log likelihood via sampling

I'm working my way through a simple variational inference from scratch. For that, I assume that z denotes the probability of a coin showing ...
1 vote
1 answer
320 views

How does the VAE learn a joint distribution?

I found the following paragraph from An Introduction to Variational Autoencoders sounds relevant, but I am not fully understanding it. A VAE learns stochastic mappings between an observed $\mathbf{x}$...
  • 203
0 votes
0 answers
25 views

Tensorflow Probability Implementation of Automatic Differentiation Variational Inference with Mixtures

In this paper, the authors suggest using the following loss instead of the traditional ELBO in order to train what basically is a Variational Autoencoder with a Gaussian Mixture Model instead of a ...
  • 1
3 votes
1 answer
130 views

What does the approximate posterior on latent variables, $q_\phi(z|x)$, tend to when optimising VAE's

The ELBO objective is described as follows $$ ELBO(\phi,\theta) = E_{q_\phi(z|x)}[log p_\theta (x|z)] - KL[q_\phi (z|x)||p(z)] $$ This form of ELBO includes a regularisation term in the form of the ...
2 votes
1 answer
258 views

What is the intuition behind variational inference for Bayesian neural networks?

I'm trying to understand the concept of Variational Inference for BNNs. My source is this work. The aim is to minimize the divergence between the approx. distribution and the true posterior $$\text{KL}...