# Bayes error rate formula clarification

My questions concern a particular formulation of the Bayes error rate from Wikipedia, summarized below.

For a multiclass classifier, the Bayes error rate may be calculated as follows: $$p = 1 - \sum_{C_i \ne C_{\max,x}} \int_{x \in H_i} P(C_i|x)p(x)\,dx$$ where $$x$$ is an instance, $$C_i$$ is a class into which an instance is classified, $$H_i$$ is the area/region that a classifier function $$h$$ classifies as $$C_i$$.

We are interested in the probability of misclassifying an instance, so we wish to sum up the probability of each unlikely class label (hence we want to look at $$C_i \ne C_{\max, x}$$). However, the integral is confusing me. We want to integrate an area corresponding to the probability that we choose label $$C_i$$ given $$x$$. But we drew $$x$$ from $$H_i$$, the region covered/classified by $$C_i$$, so wouldn't $$P(C_i|x) = 1$$?

I think most of my confusion will be resolved if someone can help clarify the intention of the integral. Is it to draw random samples from the total space of $$h$$ (the classifier function), and then sum the probabilities from each classified $$C_i \ne C_{\max, x}$$? How does $$x$$ exist in the outer summation before it has been sampled from $$H_i$$ in the integral?

Bayes Error Rate

For the general case of K different classes, the probability of classifing x instance correctly is:

$$\begin{equation} \label{eq1} \begin{split} P(correct) & = \sum_{i=1}^{K} p(x \in H_i, C_i) \\ & = \sum_{i=1}^{K} \int_{x \in H_i} p(x,C_i) \, dx\\ & = \sum_{i=1}^{K} \int_{x \in H_i} P(C_i|x)p(x)\,dx \\ \end{split} \end{equation}$$

where $$H_i$$ is the region where class $$i$$ has the highest posterior. So the Bayes Error Rate is:

$$\begin{equation} \label{eq2} \begin{split} P(error) & = 1 - p(correct) \\ & = 1 - \sum_{i=1}^{K} \int_{x \in H_i} P(C_i|x)p(x)\,dx \end{split} \end{equation}$$

Be careful if we drew $$x$$ from $$H_i$$, the region covered/classified by $$C_i$$, $$P(C_i|x) \ne 1$$, because that would mean that we surely predict correctly all the time. If $$x$$ belongs to a decision area, this does not imply that it belongs to the corresponding class.

• Ah, I think I get it now. We are looking at the region $H_i$ and integrating the probability that our $x$ would occur (given class $C_i$) over the entire region. Does that sound correct? I also note that you wrote "...where $C_i$ is the region...", but isn't $C_i$ supposed to be a class label? Jun 5 '20 at 3:06
• Yes. Fixed that Jun 5 '20 at 9:30