1
$\begingroup$

If I understand correctly, the KL divergence is a measure of information loss between a ground truth distribution $P$ and a predicted distribution $Q$, and the Jensen-Shannon divergence is the mean of the KL Divergences of 2 cases

  1. Predicted distribution is mean of $P$ and $Q$, and ground truth is $P$

  2. Predicted distribution is mean of $P$ and $Q$, and ground truth is $Q$

Since KL divergence can be easily interpreted as information loss in $Q$ relative to $P$, what can JS divergence interpret-ably represent? I cannot see any use cases of these measures unless there are two distributions to compare. Is there any other problem where I could use them as loss functions other than generation problems? If so how, and why?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.