# Does the performance of a model increase if dropout is disabled at evaluation time?

I know dropout layers are used in neural networks during training to provide a form of regularisation in an attempt to mitigate over-fitting.

Would you not get an increased fitness if you disabled the dropout layers during evaluation of a network?

Dropout is usually disabled at test (or evaluation) time. For example, in Keras, dropout is disabled at evaluation time by default, although you can enable it, if you need to (see below). The purpose of dropout is to decorrelate the units (or feature detectors) so that they learn more robust representations of the data (i.e. a form of regularisation).

However, there's also Monte Carlo (MC) dropout, i.e., you train the network with dropout and you also use dropout at test time in order to get stochastic outputs (i.e. you will get different outputs, for different forward passes, given the same inputs). MC dropout is an approximation of Bayesian inference in deep Gaussian processes, which means that MC dropout is roughly equivalent to a Bayesian neural network.

Does the performance of a model increase if dropout is disabled at evaluation time?

Yes, possibly. However, MC dropout provides an uncertainty measure, which can be useful in certain scenarios (e.g. medical scenarios), where a point estimate (i.e. a single prediction or classification) is definitely not appropriate, but you also need a measure of the uncertainty or confidence of the predictions.

Dropout is a technique that helps to avoid overfitting during training. That is, one can use dropout only for training.

units may change in a way that they fix up the mistakes of the other units. This may lead to complex co-adaptations. This, in turn, leads to overfitting because these co-adaptations do not generalize to unseen data.

If you want to evaluate your model, you should turn off all dropout layers. For example, PyTorch's model.eval() does this work.