When adding dropout to a neural network, we are randomly removing a fraction of the connections (setting those weights to zero for that specific weight update iteration). If the dropout probability is $p$, then we are effectively training with a neural network of size $(1−p)N$, where $N$ is the total number of units in the neural network.
Using this logic, there is no limit how big I can make a network, as long as I proportionately increase dropout, I can always effectively train with the same sized network, and thereby just increasing the number of "independent" models working together, making a larger ensemble model. Thereby improving generalization of the model.
For example, if a network with 2 units already achieves good results in the training set (but not in unseen data -i.e validation or test sets-), also a network with 4 units + dropout 0.5 (ensemble of 2 models), and also a network with 8 units + dropout 0.75 (ensemble of 4 models)... and also a network with 1000 units with a dropout of 0.998 (ensemble of 500 models)!
In practice, it is recommended to keep dropout at $0.5$, which advises against the approach mentioned above. So there seem to be reasons for this.
What speaks against blowing up a model together with an adjusted dropout parameter?