So after a, say, 5-fold CV, you are left with 5 models, each trained on 80% of the data. You now want to have the best model possible, i.e. train it on all data. In order to save computation time, can you 'safely' train one of the 5 models on its respective validation data for the same amount of epochs and call it a day? Or do you need to train the entire model from scratch?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
The purpose of k-fold is to perform hyperparameter tuning more than "finding the best model". We want the hyperparameters of the model to be able to generalize to all folds of the data. When you finish k-fold CV, in practice, you have to train a model from scratch again with the training + validation data, however, it is still not a bad idea to use all 5 models, and then averaging predictions on all models.
