Questions tagged [kl-divergence]

For questions related to the Kullback–Leibler (KL) divergence, which is a measure (that is not a metric, but it is pre-metric, because it does not satisfy all properties of metrics, i.e. it is not symmetric) of divergence (or distance) between two probability measures (density functions, or mass functions), which is commonly used in many machine learning settings, e.g. in the context of variational auto-encoders (VAES).

Filter by
Sorted by
Tagged with
0 votes
0 answers
20 views

VAE - Which loss to optimize for?

Regarding hyperparameter optimization for VAEs. Should you optimize for the reconstruction loss, or the complete ELBO (- KL divergence + reconstruction loss)? My thought is that it probably depends on ...
0 votes
1 answer
36 views

Distance between policies that are not probability distributions

This question asks if there is a way to measure distance between policies that are in fact probability distributions. In the case of continuous control with deterministic policies where they take a ...
0 votes
0 answers
15 views

Why did the authors of D2GAN propose two discriminators using KL and reverse KL divergence losses instead of one discriminator using JS divergence

I have stumbled upon the D2GAN paper as part of my research and I am finding myself extremely confused by the fact that instead of using the JS divergence to capture both KL and its reverse, they ...
  • 101
1 vote
0 answers
19 views

How to compare different trajecories in a Markov Decision Process

I realize that my question is a bit fuzzy and I am sorry for that. If needed, I will try to make it more rigorous and precice. Let $\mathcal{M}$ be a Markov Decision Process, with state space $\...
  • 173
0 votes
0 answers
36 views

How to calculate KL-divergence?

My problem setting is that KL-divergence is between two policies based on the data of one trajectory. As two ways to be shown below: $KL(\theta_{1}, \theta_{2}) = \sum\limits_{k=1}^{N}\pi_{\theta_{1}}...
1 vote
0 answers
424 views

What is the most suitable measure of the distance between two VAE's latent spaces?

The problem I'm trying to solve is as follows. I have two separate domains, where inputs do not have the same dimensions. However, I want to create a common feature space between both domains using ...
2 votes
1 answer
70 views

How is this statement from a TensorFlow implementation of a certain KL-divergence formula related to the corresponding formula?

I am trying to understand a certain KL-divergence formula (which can be found on page 6 of the paper Evidential Deep Learning to Quantify Classification Uncertainty) and found a TensorFlow ...
  • 33
0 votes
1 answer
344 views

KL divergence coefficient update doesn't make sense in RLlib's PPO implementation

I am using RLlib (Ray 1.4.0)'s implementation of PPO for a multi-agent scenario with continuous actions, and I find that the loss includes the KL divergence penalty term, apart from the surrogate loss,...
  • 223
1 vote
0 answers
381 views

Why does the VAE using a KL-divergence with a non-standard mean does not produce good images?

I know I can make a VAE do generation with a mean of 0 and std-dev of 1. I tested it with the following loss function: ...
  • 43
7 votes
2 answers
1k views

How is this Pytorch expression equivalent to the KL divergence?

I found the following PyTorch code (from this link) -0.5 * torch.sum(1 + sigma - mu.pow(2) - sigma.exp()) where mu is the mean ...
3 votes
1 answer
181 views

How do you calculate KL divergence on a three-dimensional space for a Variational Autoencoder?

I'm trying to implement a variational auto-encoder (as seen in Section 3.1 here: https://arxiv.org/pdf/2004.06271.pdf). It differs from a traditional VAE because it encodes its input images to three-...
7 votes
2 answers
1k views

Why is KL divergence used so often in Machine Learning?

The KL Divergence is quite easy to compute in closed form for simple distributions -such as Gaussians- but has some not-very-nice properties. For example, it is not symmetrical (thus it is not a ...
3 votes
1 answer
310 views

Are there some notions of distance between two policies?

I want to determine some distance between two policies $\pi_1 (a \mid s)$ and $\pi_2 (a \mid s)$, i.e. something like $\vert \vert \pi_1 (a \mid s) - \pi_2(a \mid s) \vert \vert$, where $\pi_i (a\mid ...
  • 287
4 votes
1 answer
1k views

What is the impact of scaling the KL divergence and reconstruction loss in the VAE objective function?

Variational autoencoders have two components in their loss function. The first component is the reconstruction loss, which for image data, is the pixel-wise difference between the input image and ...
  • 151
3 votes
2 answers
563 views

When should one prefer using Total Variational Divergence over KL divergence in RL

In RL, both the KL divergence (DKL) and Total variational divergence (DTV) are used to measure the distance between two policies. I'm most familiar with using DKL as an early stopping metric during ...
  • 513
3 votes
1 answer
340 views

What is the reason for mode collapse in GAN as opposed to WGAN?

In this article I am reading: $D_{KL}$ gives us inifity when two distributions are disjoint. The value of $D_{JS}$ has sudden jump, not differentiable at $\theta=0$. Only Wasserstein metric provides ...
  • 131
3 votes
1 answer
1k views

Why does the KL divergence not satisfy the triangle inequality?

The KL divergence is defined as $$D_{KL}=\sum_i p(x_i)log\left(\frac{p(x_i)}{q(x_i)}\right)$$ Why does $D_{KL}$ not satisfy the triangle inequality? Also, can't you make it satisfy the triangle ...
5 votes
1 answer
705 views

Why is the evidence equal to the KL divergence plus the loss?

Why is the equation $$\log p_{\theta}(x^1,...,x^N)=D_{KL}(q_{\theta}(z|x^i)||p_{\phi}(z|x^i))+\mathbb{L}(\phi,\theta;x^i)$$ true, where $x^i$ are data points and $z$ are latent variables? I was ...
5 votes
1 answer
808 views

Why is the Jensen-Shannon divergence preferred over the KL divergence in measuring the performance of a generative network?

I have read articles on how Jensen-Shannon divergence is preferred over Kullback-Leibler in measuring how good a distribution mapping is learned in a generative network because of the fact that JS-...
  • 1,369
2 votes
1 answer
206 views

How does the Kullback-Leibler divergence give "knowledge gained"?

I'm reading about the KL divergence on Wikipedia. I don't understand how the equation gives "information gained" as it says in the "Interpretations" section Expressed in the ...
  • 351
5 votes
2 answers
3k views

What are the advantages of the Kullback-Leibler over the MSE/RMSE?

I've recently encountered different articles that are recommending to use the KL divergence instead of the MSE/RMSE (as the loss function), when trying to learn a probability distribution, but none of ...
  • 1,108
18 votes
1 answer
4k views

Why has the cross-entropy become the classification standard loss function and not Kullback-Leibler divergence?

The cross-entropy is identical to the KL divergence plus the entropy of the target distribution. The KL divergence equals zero when the two distributions are the same, which seems more intuitive to me ...