The divergence between two probability distributions is used in calculating the difference between the true distribution and generated distribution. These divergence metrics are used in loss functions.

Some divergence metrics that are generally used in literature are:

  1. Kullback-Leibler Divergence
  2. Jensen–Shannon divergence
  3. f-divergence
  4. Wasserstein distance

Some other divergence measures include:

  1. Squared Hellinger distance
  2. Jeffreys divergence
  3. Chernoff's $\alpha-$divergence
  4. Exponential divergence
  5. Kagan's divergence
  6. $(\alpha, \beta)-$product divergence
  7. Bregman divergence

I think some naive divergence measures include

  1. Least-squares divergence
  2. Absolute deviation

Along with these, are there any other divergence measures available to compute the distance between the true probability distribution and estimated probability distribution in artificial intelligence?

  • $\begingroup$ You're already listing many measures. Probably all the major ones used in the literature that I know too. Anyway, keep in mind that our site is not appropriate for asking questions that require very long answers that require a lot of effort and time to write. So, it may be a good idea to reformulate your question that you provide more details about why you're interested in more measures and maybe ask for a reference that covers them (rather than asking for us to enumerate them), if you're interested in an extensive list. In that case, you may want to use the tag reference-request. $\endgroup$
    – nbro
    Commented Aug 8, 2021 at 12:29
  • 1
    $\begingroup$ Having said that, in this answer, I provide a link to a paper that talks about the topic, but I don't remember which measures are listed there (maybe you already list all the ones mentioned there too). $\endgroup$
    – nbro
    Commented Aug 8, 2021 at 12:30
  • 1
    $\begingroup$ @nbro Thanks... I rephrased, I think now even any single metric is eligible as an answer. $\endgroup$
    – hanugm
    Commented Aug 8, 2021 at 13:44


You must log in to answer this question.

Browse other questions tagged .