2
$\begingroup$

According to Wikipedia

Dropout is a regularization technique for reducing overfitting in neural networks

My neural network is simple enough and does not overfit.

Can the addition of dropout, in a non-overfitting neural network, increase accuracy? Even if I increase the complexity of the neural network?

$\endgroup$
2
$\begingroup$

Can the addition of dropout, in a non-overfitting neural network, increase accuracy?

Yes, maybe.

Even if I increase the complexity of the neural network?

Yes, maybe.

As always when making changes to ML algorithms, you need to test carefully to see if your changes have made an improvement. There are very few theories in non-linear machine learning models that make solid guarantees of results. One general difference you should note is that training a network with dropout will take longer (more epochs) than training a similar network without dropout, to reach the same levels of accuracy.

However, as well as the regularisation effect of dropout, it shares some behaviour with ensemble techniques such as bagging. Dropout effectively trains many sub-networks (that share weights) on different samples of the training data. This pseudo-ensemble effect can boost accuracy, and other success metrics. This is not a guaranteed effect, but it does happen in practice.

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.