In Monte Carlo Dropout (MCD), I know that I should enable dropout during training and testing, then get multiple predictions for the same input $x$ by performing multiple forward passes with $x$, then, for example, average these predictions.

Let's suppose I want to fine-tune a pre-trained model and get MCD uncertainty estimations, how should I add dropout layers?

  • on the fully-connected layers;
  • after every convolutional layer.

I've read some papers and implementations where one applies dropout at fully-connected layers only, using a pre-trained model. However, when using a custom model, usually one adds dropout after every convolutional layer. This work builds two configurations:

  • dropout on the fully-connected layers;
  • dropout after resnet blocks.

The first configuration performs better, but I'm unsure if this is an actual uncertainty estimation from resnet. The results show that there is a correlation between high uncertainty predictions and incorrect predictions. So, would this be a good way of estimating uncertainty? My shot is "yes", because even though there are no nodes being sampled from the backbone, the sampling in the fully-connected layer forces a smooth variation in the backbone, generating a low-variance ensemble. But, I'm quite a beginner on MCD so, any help would be appreciated.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.