1
$\begingroup$

My neural network is simple enough and does not overfit.

Dropout is a regularization technique for reducing overfitting in neural networks

From Wikipedia

Adding Dropout in a non-overfitting network can increase accuracy ? Even if I increase the complexity of the network ?

$\endgroup$
2
$\begingroup$

Adding Dropout in a non-overfitting network can increase accuracy ?

Yes, maybe.

Even if I increase the complexity of the network ?

Yes, maybe.

As always when making changes to ML algorithms, you need to test carefully to see if your changes have made an improvement. There are very few theories in non-linear machine learning models that make solid guarantees of results. One general difference you should note is that training a network with dropout will take longer (more epochs) than training a similar network without dropout, to reach the same levels of accuracy.

However, as well as the regularisation effect of dropout, it shares some behaviour with ensemble techniques such as bagging. Dropout effectively trains many sub-networks (that share weights) on different samples of the training data. This pseudo-ensemble effect can boost accuracy, and other success metrics. This is not a guaranteed effect, but it does happen in practice.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.