Why is my Keras prediction always close to 100% for one image class?

Question

I am using Keras (on top of TF 2.3) to train an image classifier. In some cases I have more than two classes, but often there are just two classes (either "good" or "bad"). I am using the tensorflow.keras.applications.VGG16 class as base model with a custom classifier on top, like this:

input_layer = layers.Input(shape=(self.image_size, self.image_size, 3), name="model_input")
base_model = VGG16(weights="imagenet", include_top=False, input_tensor=input_layer)
model_head = base_model.output
model_head = layers.AveragePooling2D(pool_size=(4, 4))(model_head)
model_head = layers.Flatten()(model_head)
model_head = layers.Dense(256, activation="relu")(model_head)
model_head = layers.Dropout(0.5)(model_head)
model_head = layers.Dense(len(self.image_classes), activation="softmax")(model_head)

As you can see in the last (output) layer I am using a softmax activation function. Then I compile the whole model with the categorical_crossentropy loss function and train with one-hot-encoded image data (labels).

All in all the model performs quite well, I am happy with the results, I achieve over 99% test and validation accuracy with our data set. There is one thing I don't understand though:

When I call predict() on the Keras model and look at the prediction results, then these are always either 0 or 1 (or at least very, very close to that, like 0.000001 and 0.999999). So my classifier seems to be quite sure whether an image belongs to either class "good" or "bad" (for example, if I am using only two classes). I was under the assumption, however, that usually these predictions are not that clear, more in terms of "the model thinks with a probability of 80% that this image belongs to class A" - but as said in my case it's always 100% sure.

Any ideas why this might be the case?

Answer 1

Traditional neural networks can be over-confident (i.e. give a probability close to $0$ or $1$) even when they are wrong, so you should not interpret the probability that it produces as a measure of uncertainty (i.e. as a measure of how much it is confident that the associated predicted class is the correct one), as that it is essentially wrong. See this and this answers for more details about this.

Given that this overconfidence is not desirable in many scenarios (such as healthcare, where doctors also want to know how confident the model is about its predictions, in order to decide whether to give a certain medication to the patient or not), the ML community has been trying to incorporate uncertainty quantification/estimation in neural networks. If you are interested in this topic, you could read the paper Weight Uncertainty in Neural Network (2015) by Blundell et al., which proposes a specific type of Bayesian neural network, i.e. a neural network that models the uncertainty over the actual values of the weights, from which we may also quantify/estimate the uncertainty about the inputs. This paper should not be too difficult to read if you are already familiar with the details of variational-autoencoders.

So, the answer to your question is: yes, it's possible that the output probability is close to $1$ because neural networks can be over-confident. (I am assuming that the values returned by tf.keras's predict method are probabilities: I don't remember anymore, so I assumed that you did not make any mistake).

A similar question was already asked in the past here. The accepted answer should provide more details about different types of uncertainty and solutions.

Answer 2

Without more details about the nature of the dataset, it is impossible to know for sure. However, here are a few likely causes:

1. You were calling predict on training data, not testing data. The network will be a lot more sure about images that it trained on than on images it has never seen before.

2. Your model overfit the data. This can happen when you use an overly complex model on a small dataset. You may want to experiment with regularization.

3. You were looking at too small a sample of images. Did you run predict on every image, or just a few? If the latter, it is possible you just picked a sample that the network is very confident about.