1
$\begingroup$

I've been working on a lot of simple resnet18 binary classifiers lately and I've started to notice that the probability distributions are often skewed one way or the other. This figure shows one such example. The red and blue color code the negative and positive ground truths respectively. And the bottom axis is the output prediction of the binary classifier (sigmoid activated output neuron). Notice how the red is more bunched towards 0, but the blue has quite some spread.

enter image description here

At first I began to reason this to myself with arguments like "well the positive clues in the image have a small footprint, so they are hard to find, therefore the model should be unsure about positives more of the time."

Later I found oppositely skewed distributions and tried to say "well the positive clues in the image have a small footprint, so it might be easy to confuse some other things for the positive clues, therefore the model should be unsure about negatives more of the time"

You can see where I'm going with this. It took me training up quite a few models like this in a short amount of time to realise I was kidding myself. Even the exact same architecture and similar dataset may produce a different skew over different training runs. And if you think about it, negative probability is just the complement of positive probability, so any argument you make in favor of one over the other can be easily reversed.

So what's influencing this skew? Why is there a skew at all? If there's a skew, is it because of something "real", or is it just random?

These all may seem like philosophical questions, but they have great practical significance. Because that skew basically tells me where I should put my decision threshold in production level inference!

$\endgroup$
1
$\begingroup$

Yes, due to this issue, you should use temperature scaling after training your model. It will calibrate your probability and you will start to get the same kind of distributions. Here are a good article along with implementation on it.

$\endgroup$
2
  • $\begingroup$ Thanks! This is a useful answer and checks the "practical" box. I'm also wondering a lot about the data science behind why this is the case. I had previously thought that logistic regression (which is essentially what's happening on top of the CNN) is supposed to give calibrated probabilities. In fact, that's what Platt Scaling is really doing. $\endgroup$ – Alexander Soare Feb 26 at 8:50
  • $\begingroup$ The reason each time, you get a different model is because of the entropy involved in creating the new model (initialization). Hence, reproducibility has come into scene strongly in DL. But, even if we set the random seed at every possible level, some amount of entropy is still retained. It is that entropy which leads to a different solution every single time. A lot of regularization may help with that, one should experiment. So, every time, you end up with a different minima in the loss landscape. $\endgroup$ – Abhishek Verma Feb 26 at 9:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.