1
$\begingroup$

We know that in machine learning the dataset is divided into 3 parts: training data, validation data and test data.

On the other hand, K-fold cross-validation is defined as follows:

the dataset is divided into K number of different sectors. One section is used for testing and the rest for training. The results of these K-iterative tests are then averaged to get the final accuracy.

What happens to validation dataset in K-fold cross-validation? Is there such a dataset?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Before doing the $k$-fold cross-val you divide all your dataset into train and test splits (e.g., 80-20 in proportion), then the $k$-fold cross validation is performed on the training split as follows:

enter image description here The training data is split into $k$ folds, where at each training iteration a fold is held out for validation: so $k-1$ folds are effectively used for training. The process repeats $k$ times, each time using a different fold for validation.

The $k$-fold cross-val approach is useful when your dataset is small, also it can be used as an easy way that provides $k$ (slightly different) models to build an ensemble (since each iteration trains a new model on a different split of the training data), which is supposed to generalize better. Moreover, since you have $k$ models you can use them to estimate the variance of their predictions, which can be used as a way to estimate uncertainty.

Improve this answer
$\endgroup$
3
  • $\begingroup$ I think it is like this. You divide the dataset into K-folds. 1 fold is used for test. Then, you divide the remaining K-1 folds into training and validation based on a chosen percentage. $\endgroup$
    – DSPinfinity
    Commented Oct 8, 2023 at 18:38
  • $\begingroup$ This is a nice description of k-fold CV, but I don't think this answers the actual question. Do we need to differentiate between testing a validation datasets in k-fold CV? $\endgroup$
    – nbro
    Commented Oct 9, 2023 at 22:05
  • $\begingroup$ @nbro Well, the fact is that in classical ML there is no notion of validation set, which instead is present in modern ML and deep learning. In the first case, you perform K-fold cross-val on the entire dataset without the train-test splitting beforehand: so one fold is used for testing. But in general, especially when you need to tune hyper-parameters, performing k-fold cross-val on the training-split, and after having the real evaluation on the test-set is the safest option to not underestimate the generalization error. $\endgroup$
    – Luca Anzalone
    Commented Oct 12, 2023 at 13:58

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .