2
$\begingroup$

In Monte Carlo Dropout (MCD), I know that I should enable dropout during training and testing, then get multiple predictions for the same input $x$ by performing multiple forward passes with $x$, then, for example, average these predictions.

Let's suppose I want to fine-tune a pre-trained model and get MCD uncertainty estimations, how should I add dropout layers?

  • on the fully-connected layers;
  • after every convolutional layer.

I've read some papers and implementations where one applies dropout at fully-connected layers only, using a pre-trained model. However, when using a custom model, usually one adds dropout after every convolutional layer. This work builds two configurations:

  • dropout on the fully-connected layers;
  • dropout after resnet blocks.

The first configuration performs better, but I'm unsure if this is an actual uncertainty estimation from resnet. The results show that there is a correlation between high uncertainty predictions and incorrect predictions. So, would this be a good way of estimating uncertainty? My shot is "yes", because even though there are no nodes being sampled from the backbone, the sampling in the fully-connected layer forces a smooth variation in the backbone, generating a low-variance ensemble. But, I'm quite a beginner on MCD so, any help would be appreciated.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.