5
$\begingroup$

Hyperparameter tuning is the process of selecting the optimal hyperparameters for an ANN.

Now, my guess is that, if we have sufficient data (say, 1.4 million for, say, 6 features), the model can be optimally trained and we don't need a hyperparameter tuner (like Keras-Tuner), because, while training, the data itself will optimize the model.

Do we need a hyperparameter tuner if we have a sufficient number of random data for training our ANN model?

$\endgroup$
6
$\begingroup$

Unfortunately, even with large amounts of training data, hyperparameter choices can strongly influence the performance of a trained model.

What you can usually drop when you have large amounts of training data is regularisation. If your training examples cover the function space you are learning really well, then it is harder to overfit the training data. Regularisation choices are also hyperparameters, so you can save some search space and time by ignoring them.

Do we need a hyperparameter tuner if we have a sufficient number of random data for training our ANN model?

Having lots of data may mean that you can use simpler "brute force" architectures and designs, and that the end result is robust over wider range of hyperparameter choices.

You may still want to tune hyperparameters at least a little though. This tuning can be tedious to drive by manual edits and re-tries, which is where an automated tuner can help.

$\endgroup$
3
$\begingroup$

You don't NEED a hyperparameter tuner, but it can help in various situations. For example, if your model is not training well, perhaps using a tuner can help.

It's hard to say in which hyperparameters you would be turning over in your specific model, but for some specific hyperparameters if you choose a bad value your model won't learn or diverge. Take for example the learning rate, if you pick a value too high it will overshoot minimums and the error might constantly grow (divergence), or if you pick a value too low it will get stuck in a local minimum and not be able to continue learning. You can have the world's largest dataset, but if you don't pick the correct range for your learning rate, the model will not properly learn.

In general, if you're not confident about specific hyperparameters ranges, then hyperparameter tuning can be a helpful tool, regardless of your dataset size

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.