I've been working on a lot of simple resnet18 binary classifiers lately and I've started to notice that the probability distributions are often skewed one way or the other. This figure shows one such example. The red and blue color code the negative and positive ground truths respectively. And the bottom axis is the output prediction of the binary classifier (sigmoid activated output neuron). Notice how the red is more bunched towards 0, but the blue has quite some spread.
At first I began to reason this to myself with arguments like "well the positive clues in the image have a small footprint, so they are hard to find, therefore the model should be unsure about positives more of the time."
Later I found oppositely skewed distributions and tried to say "well the positive clues in the image have a small footprint, so it might be easy to confuse some other things for the positive clues, therefore the model should be unsure about negatives more of the time"
You can see where I'm going with this. It took me training up quite a few models like this in a short amount of time to realise I was kidding myself. Even the exact same architecture and similar dataset may produce a different skew over different training runs. And if you think about it, negative probability is just the complement of positive probability, so any argument you make in favor of one over the other can be easily reversed.
So what's influencing this skew? Why is there a skew at all? If there's a skew, is it because of something "real", or is it just random?
These all may seem like philosophical questions, but they have great practical significance. Because that skew basically tells me where I should put my decision threshold in production level inference!