4
$\begingroup$

I very often applied a grid search to tune the parameters of my supervised model. I have the feeling that parameter tuning will eventually (very often) lead to overfitting? Is this crazy to say?

Is there a way that we can apply grid search in such a way that it will not overfit?

$\endgroup$
3
$\begingroup$

Yes. Usually you would use cross validation to avoid overfitting during parameter tuning. If your dataset is large enough, and you don't try too many parameter combinations, this will work well, because to "get lucky" and overfit, a parameter combination will need to work very well on many variations of the problem, which is less likely than working well on just one set of data.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.