While training a standard VAE, we assume that the prior on the latent variable Z is the standard gaussian and we use KL divergence to push the posterior as close as possible to the standard gaussian. Why not assume any other gaussian as the prior? What are the intuitive reasons for this?
$\begingroup$
$\endgroup$
1
-
$\begingroup$ A good reason is we normalize data before the training process! $\endgroup$– OmGCommented Aug 25, 2021 at 13:30
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Simply, it is just a design choice.
Isotropic gaussian is one of the easiest density to work with. It has an easy-to-compute likelihood and easily reparameterizable.
You are free to use other distribution, but might face computational or implementation hurdles.
