My questions concern a particular formulation of the Bayes error rate from Wikipedia, summarized below.

For a multiclass classifier, the Bayes error rate may be calculated as follows: $$p = 1 - \sum_{C_i \ne C_{\max,x}} \int_{x \in H_i} P(C_i|x)p(x)\,dx$$ where $x$ is an instance, $C_i$ is a class into which an instance is classified, $H_i$ is the area/region that a classifier function $h$ classifies as $C_i$.

We are interested in the probability of misclassifying an instance, so we wish to sum up the probability of each unlikely class label (hence we want to look at $C_i \ne C_{\max, x}$).

However, the integral is confusing me. We want to integrate an area corresponding to the probability that we choose label $C_i$ given $x$. But we drew $x$ from $H_i$, the region covered/classified by $C_i$, so wouldn't $P(C_i|x) = 1$?

I think most of my confusion will be resolved if someone can help clarify the intention of the integral.

Is it to draw random samples from the total space of $h$ (the classifier function), and then sum the probabilities from each classified $C_i \ne C_{\max, x}$? How does $x$ exist in the outer summation before it has been sampled from $H_i$ in the integral?


1 Answer 1


Bayes Error Rate

For the general case of K different classes, the probability of classifing x instance correctly is:

\begin{equation} \label{eq1} \begin{split} P(correct) & = \sum_{i=1}^{K} p(x \in H_i, C_i) \\ & = \sum_{i=1}^{K} \int_{x \in H_i} p(x,C_i) \, dx\\ & = \sum_{i=1}^{K} \int_{x \in H_i} P(C_i|x)p(x)\,dx \\ \end{split} \end{equation}

where $H_i$ is the region where class $i$ has the highest posterior. So the Bayes Error Rate is:

\begin{equation} \label{eq2} \begin{split} P(error) & = 1 - p(correct) \\ & = 1 - \sum_{i=1}^{K} \int_{x \in H_i} P(C_i|x)p(x)\,dx \end{split} \end{equation}

Be careful if we drew $x$ from $H_i$, the region covered/classified by $C_i$, $P(C_i|x) \ne 1$, because that would mean that we surely predict correctly all the time. If $x$ belongs to a decision area, this does not imply that it belongs to the corresponding class.

  • $\begingroup$ Ah, I think I get it now. We are looking at the region $H_i$ and integrating the probability that our $x$ would occur (given class $C_i$) over the entire region. Does that sound correct? $\endgroup$ Commented Jun 5, 2020 at 3:06

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .