# Understanding how to calculate $P(x|c_k)$ for the Bernoulli naïve Bayes classifier

I'm looking at the Bernoulli naïve Bayes classifier on Wikipedia and I understand Bayes theorem along with Gaussian naïve Bayes, however when looking at how $$P(x|c_k)$$ is calculated I don't understand it. The Wikipedia page says its calculated as follows $$P(x|c_k) = \prod^{n}_{i=1} p^{x_i}_{ki} (1-p_{ki})^{(1-x_i)}$$.

They mention that $$p_{ki}$$ is the probability of class $$c_k$$ generating the term $$x_i$$, does that mean $$P(x|c_k)$$? Because if so then that doesn't make sense since to calculate that we need to have calculated it already. So what is $$p_{ki}$$?

And in the first part, after the product symbol, are they raising this probability to the power pf $$x_i$$ or does that again just mean 'probability of class $$c_k$$ generating the term $$x_i$$'?

I also don't understand the intuition behind why or how this calculates $$P(x|c_i)$$.

• Isn't $x$ a vector $[x_1, \dots, x_n]$? – nbro May 3 at 16:10
• @nbro Yes it's a feature vector although it can also be a matrix for larger data sets where the amount of rows is the 'number of samples', and the amount of columns is the 'number of features'. – Aguy May 3 at 16:22
• Therefore, that probability you're analysing can also be written as $P(x_1, \dots, x_n \mid c_k)$. Does this make things clearer? Now, what happens if you assume the observations are independent of each other? – nbro May 3 at 16:23
• @nbro I understand that. The problem I was facing is understanding what $p_{ki}$ is. And when in the formula they write $p_{ki}^{x_i}$ are they raising it to the power of $x_i$ or is it just saying that we're calculating $p_{ki}$ given $x_i$? – Aguy May 3 at 16:26
• It's been a while since I had to deal with naive Bayes. so I cannot help you more unless I read about it again (maybe I'll do that later), but have a look at the binomial distribution. The formulas look very similar. Anyway, I am pretty sure that this is an exponentiation operation, otherwise, what would the exponent $1 - x_i$ mean? In any case, $x_i$ must be encoded as a number. Maybe they mean $1 - i$ (rather than $1 - x_i$). Similarly, maybe they meant $p_{ki}^i$. – nbro May 3 at 16:31

Bernoulli naïve Bayes

$$P(x|c_k) = \prod^{n}_{i=1} p^{x_i}_{ki} (1-p_{ki})^{(1-x_i)}$$

Let's examine the example of document classification.
Let K different text classes and n different terms that our vocabulary contains. $$x_i$$ are boolean variables (0, 1) expressing if the $$i^{th}$$ term exists in document x. x is a vector of dimension n.

$$P(x|c_k)$$ is the probability that given the class k, document x to be generated. The equation uses a common trick to represent a multivariate Bernoulli event model, taking into account that when $$x_i = 1$$, then $$1 - x_i = 0$$ and inversely. In other words, for each term it takes the probability that the document does contain this term or it does not.

$$p_{ki}$$ is the probability of class $$c_k$$ generating the term $$x_i$$, that is it could be the prior probability a document that belongs to k class contains this term of the vocabulary.