# Questions tagged [kl-divergence]

For questions related to the Kullback–Leibler (KL) divergence, which is a measure (that is not a metric, but it is pre-metric, because it does not satisfy all properties of metrics, i.e. it is not symmetric) of divergence (or distance) between two probability measures (density functions, or mass functions), which is commonly used in many machine learning settings, e.g. in the context of variational auto-encoders (VAES).

5 questions
Filter by
Sorted by
Tagged with
46 views

### What is the impact of scaling the KL divergence and reconstruction loss in the VAE objective function?

Variational autoencoders have two components in their loss function. The first component is the reconstruction loss, which for image data, is the pixel-wise difference between the input image and ...
40 views

### When should one prefer using Total Variational Divergence over KL divergence in RL

In RL, both the KL divergence (DKL) and Total variational divergence (DTV) are used to measure the distance between two policies. I'm most familiar with using DKL as an early stopping metric during ...
54 views

### What is the reason for mode collapse in GAN as opposed to WGAN?

In this article I am reading: $D_{KL}$ gives us inifity when two distributions are disjoint. The value of $D_{JS}$ has sudden jump, not differentiable at $\theta=0$. Only Wasserstein metric provides ...
Why is the equation $$\log p_{\theta}(x^1,...,x^N)=D_{KL}(q_{\theta}(z|x^i)||p_{\phi}(z|x^i))+\mathbb{L}(\phi,\theta;x^i)$$ true, where $x^i$ are data points and $z$ are latent variables? I was ...