My questions concern a particular formulation of the Bayes error rate from Wikipedia, summarized below.

For a multiclass classifier, the Bayes error rate may be calculated as follows: $$p = 1 - \sum_{C_i \ne C_{\max,x}} \int_{x \in H_i} P(C_i|x)p(x)\,dx$$ where $x$ is an instance, $C_i$ is a class into which an instance is classified, $H_i$ is the area/region that a classifier function $h$ classifies as $C_i$.

We are interested in the probability of misclassifying an instance, so we wish to sum up the probability of each unlikely class label (hence we want to look at $C_i \ne C_{\max, x}$). However, the integral is confusing me. We want to integrate an area corresponding to the probability that we choose label $C_i$ given $x$. But we drew $x$ from $H_i$, the region covered/classified by $C_i$, so wouldn't $P(C_i|x) = 1$?

I think most of my confusion will be resolved if someone can help clarify the intention of the integral. Is it to draw random samples from the total space of $h$ (the classifier function), and then sum the probabilities from each classified $C_i \ne C_{\max, x}$? How does $x$ exist in the outer summation before it has been sampled from $H_i$ in the integral?


Bayes Error Rate

For the general case of K different classes, the probability of classifing x instance correctly is:

\begin{equation} \label{eq1} \begin{split} P(correct) & = \sum_{i=1}^{K} p(x \in H_i, C_i) \\ & = \sum_{i=1}^{K} \int_{x \in H_i} p(x,C_i) \, dx\\ & = \sum_{i=1}^{K} \int_{x \in H_i} P(C_i|x)p(x)\,dx \\ \end{split} \end{equation}

where $H_i$ is the region where class $i$ has the highest posterior. So the Bayes Error Rate is:

\begin{equation} \label{eq2} \begin{split} P(error) & = 1 - p(correct) \\ & = 1 - \sum_{i=1}^{K} \int_{x \in H_i} P(C_i|x)p(x)\,dx \end{split} \end{equation}

Be careful if we drew $x$ from $H_i$, the region covered/classified by $C_i$, $P(C_i|x) \ne 1$, because that would mean that we surely predict correctly all the time. If $x$ belongs to a decision area, this does not imply that it belongs to the corresponding class.

  • $\begingroup$ Ah, I think I get it now. We are looking at the region $H_i$ and integrating the probability that our $x$ would occur (given class $C_i$) over the entire region. Does that sound correct? I also note that you wrote "...where $C_i$ is the region...", but isn't $C_i$ supposed to be a class label? $\endgroup$ – EntangledLoops Jun 5 '20 at 3:06
  • $\begingroup$ Yes. Fixed that $\endgroup$ – ddaedalus Jun 5 '20 at 9:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.