Timeline for How does using the ELBO in VAEs make the problem tractable?
Current License: CC BY-SA 4.0
5 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jul 18 at 18:16 | comment | added | cinch | You have constraints regarding the simple tractable p(z|x) since as posterior it must be close to its prior p(z) which is typically Gaussian. And due to the apparent conjugate prior property of both Gaussian p(z) and likelihood p(x|z), p(z|x) must be Gaussian, no simpler distribution is possible. | |
| Feb 21 at 7:44 | comment | added | Kiran Manicka | From what I am understanding the reason why p(x|z) is tractable is because we simply choose for it to tractable my modeling it with a simply tractable distribution like Gaussian. Then obviously because of p(x), p(z|x) becomes intractable and we have to approximate it with VI. Theoretically, is it possible to choose a simple tractable distribution for p(z|x) instead which would force us to VI approximation on the likelihood? | |
| Feb 21 at 6:29 | comment | added | cinch | Of course encoder is used to approximate parameters of p(z|x), similar to decoder p(x|z). As implicated in my answer p(z|x)p(x)=p(x|z)p(z) and RHS terms are both tractable, so what I mean p(z|x) is intractable (analytically) is due to intractability of p(x). If you were able to directly learn a parametric model of the evidence p(x) (an integral expression per chain rule as a GMM in my ref) from first principles such as MLE or MAP, then you can simply generate output by sampling p(x). The difficulty of p(x) is that integration in high dimensional latent space is intractable even numerically. | |
| Feb 21 at 0:29 | comment | added | Kiran Manicka | Thank you for you answer. But how can we automatically say that we can model p(x|z) with a decoder, but we can't do the same with p(z|x). Through Bayes formula isn't p(x|z) also dependent on p(x)? | |
| Feb 19 at 5:40 | history | answered | cinch | CC BY-SA 4.0 |