I have a probabilistic classifier that produces a distribution over my 3 classes - C1, C2, C3. I want to compare some new points I'm classifying to each other, to see which one is the best fit for a specific class.

for example:. for a new point X1 the classifier will output something like [0.2, 0.2, 0.6] for another new point X2 it will produce [0.2, 0.4, 0.4] so for both X1, X2 - the chosen class would be C3. Now I want to know - which of X1, X2 is a better fit to C3 I cannot simply choose the one with the highest probability for C3, because it's probability for C3 depends on it's probabilities for the other classes. X1 got 0.6 and X2 0.4, but it's possible that X2 is closer to C3 in the hyper-plane than X1, it is just less unique for C3 than X1, and therefor X1 got a higher probability.

here's a visual, in 2 dimensions:

enter image description here

X2, who got a lower probability, is clearly a better fit to the Red class then X1, which is truly unique to the Red class, but is further from the class cluster

My questions are:

  1. how do I normalize the results of a probabilistic classifier so I can compare predictions to each other?
  2. given an output of a probabilistic classier - how can I get the actual distance from the probabilities. It must be possible because there's an exact mapping between a set of probabilities to a point in the classified hyper-plan.

Thanks a lot! Amir


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.