Whenever I tune my neural network, I usually take the common approach of defining some layers with some neurons.

  • If it overfits, I reduce the layers, neurons, add dropout, utilize regularisation.

  • If it underfits, I do the other way around.

But it sometimes feels illogical doing all these. So, is there a more principled way of tuning a neural network (i.e. find the optimal number of layers, neurons, etc., in a principled and mathematical sound way), in case it overfits or underfits?

  • 1
    $\begingroup$ How is neural architecture search performed? may be helpful. $\endgroup$
    – user9947
    Mar 18 '20 at 22:17
  • 1
    $\begingroup$ I have edited your post to hopefully make it clearer. Please, make sure that it is still consistent with your original question. $\endgroup$
    – nbro
    Mar 20 '20 at 22:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.