Artificial Intelligence Stack Exchange is a question and answer site for people interested in conceptual questions about life and challenges in a world where "cognitive" functions can be mimicked in purely digital environment. It only takes a minute to sign up.
Sign up to join this community
According to Wikipedia
Dropout is a regularization technique for reducing overfitting in neural networks
My neural network is simple enough and does not overfit.
Can the addition of dropout, in a non-overfitting neural network, increase accuracy? Even if I increase the complexity of the neural network?
Can the addition of dropout, in a non-overfitting neural network, increase accuracy?
Yes, maybe.
Even if I increase the complexity of the neural network?
Yes, maybe.
As always when making changes to ML algorithms, you need to test carefully to see if your changes have made an improvement. There are very few theories in non-linear machine learning models that make solid guarantees of results. One general difference you should note is that training a network with dropout will take longer (more epochs) than training a similar network without dropout, to reach the same levels of accuracy.
However, as well as the regularisation effect of dropout, it shares some behaviour with ensemble techniques such as bagging. Dropout effectively trains many sub-networks (that share weights) on different samples of the training data. This pseudo-ensemble effect can boost accuracy, and other success metrics. This is not a guaranteed effect, but it does happen in practice.
