# Why isn't the reverse KL divergence commonly used in supervised learning?

Forward KL Divergence (also known as cross entropy loss) is a standard loss function in supervised learning problems. I understand why it is so: matching a known a trained distribution to a known distribution fits $$P \log(P/Q)$$ where $$P$$ is the known distribution.

Why isn't the reverse KL divergence commonly used in supervised learning?

• Maybe say a little more about what reverse KL divergence is. – Philip Raeisghasem Apr 5 '19 at 19:54
• Reverse KL Divergence would be $Q \log (Q/P)$ – cgo Apr 8 '19 at 0:24

A common problem in supervised learning where you will see KL-divergence used are classification tasks. Very often in those cases the data points in the training set are assumed to belong to a single class $$c_i$$ where $$i$$ is from some index set $$I$$. The class membership is represented by one-hot encoding, which corresponds to a distribution $$P$$ with $$p_i = 1$$ for the class the data point belongs to and $$p_j = 0$$ for all $$i \neq j$$. The loss function mostly used in this problems is the average categorical cross-entropy $$\langle E_P[-log(Q)]\rangle_{\rm data} = \langle H(P) - D(P || Q)\rangle_{\rm data}$$, which in the case of single-class membership reduces to the binary cross-entropy. Because only one probability in $$P$$ is non-zero in this case, the cross-entropy simplifies to $$\langle -\log(q_i)\rangle_{\rm data}$$. The reverse KL-Divergence $$D(Q || P)$$ is strictly speaking not even defined in this case, as $$Q$$ needs to be absolute continuous w.r.t. $$P$$.

There are of course cases were class membership is probabilistic also in the training data and the above argument doesn't apply in full. From a more conceptual point of view, this thread summaries nicely the relation of cross-entropy to maximizing the likelihood of the observed data under the model. Using the reverse cross-entropy would maximize the likelihood of a typical sample drawn from the model under the (empirical) distribution of the observed data.

A small side note when you read the above link is that in the above described classification problem we have a probability model for the class membership for each point in the input space, this is why the average cross-entropy is minimized.

Classification problems are just one example of supervised learning, so this answer might not fully cover your question.