Questions tagged [k-fold-cv]
For questions related to the k-fold cross-validation technique, where you split the dataset into k folds (subsets), train the model on k - 1 of these folds and test it on the remaining (test) fold; then repeat this procedure for each of the k folds, such that we compute the test performance for each fold; finally, we can average these test performances.
7
questions
0
votes
1
answer
33
views
Should we always use the usual no leakage train-val-test splt in time series?
Some of you may be familiar with the unusual split scheme used for time-series data. In short, there is a saying that one should only consider a split where the training set comes prior to the testing ...
- 361
0
votes
1
answer
55
views
Pretrain large model on single GPU [closed]
i want to pretrain some model on P100 which is provided by kaggle. Pretraining on 3 A100 is about 1.5 day. I have 2 questions:
Can I put the same seed everywhere so that the results match, train the ...
0
votes
1
answer
83
views
Why would the "improvement" be the result of random initialization, and so why should we use multiple runs?
I got this feedback for my thesis paper.
The improvement shown in the results section could be the result of random initialization. There should be multiple runs with means and standard deviations.
...
0
votes
1
answer
579
views
Is it valid to implement hyper-parameter tuning and THEN cross-validation?
I have a multi-label classification task I am solving. I have done hyperparameter tuning (with Keras Tuner) to determine the best configuration for my neural network.
Is it valid to do this (determine ...
- 161
6
votes
4
answers
1k
views
Is k-fold cross-validation more effective than splitting the dataset into training and test datasets to prevent overfitting?
I want to prevent my model from overfitting. I think that k-fold cross-validation (because it is doing this each time with different datasets) may be more effective than splitting the dataset into ...
- 579
7
votes
1
answer
322
views
What is the best measure for detecting overfitting?
I wanted to ask about the methodology of testing the ML models against overfitting. Please note that I don't mean any overfitting reducing methods like regularisation, just a measure to judge whether ...
- 1,240
1
vote
1
answer
215
views
Is it possible to combine k-fold cross-validation and oversampling for a multi-class text classification task with imbalanced data?
I am dealing with an intent classification task on an Italian customer service data set.
I've more or less 1.5k sentences and 29 classes (imbalanced).
According to the literature, a good choice is to ...
- 65