5
$\begingroup$

I want to prevent my model from overfitting. I think that k-fold cross-validation (because it is doing this each time with different datasets) may be more effective than splitting the dataset into training and test datasets to prevent overfitting, but a colleague (who has little experience in ML) says that, to prevent overfitting, the 70/30% split performs better than the k-fold cross-validation. In my opinion, k-fold cross-validation provides a reliable method to test the model performance.

Is k-fold cross-validation more effective than splitting the dataset into training and test datasets to prevent overfitting? I am not concerned with computational resources.

$\endgroup$
0
4
$\begingroup$

K-fold cross-validation is probably preferred in terms of completeness and generalization: you ensure that the system has seen the complete dataset for training. However, in deep learning this is often not feasible due to time and power constraints. They can both be used, and there is not one better than the other. It really depends on the specific case, the size of the dataset and the time and hardware available. Note that overfitting can be (partially) remedied by things such as dropout.

To be fair: it is fine to have a discussion about this with your colleagues, but as so often there is no one correct answer. If you really want proof, you can test it out and compare them. But performance-wise (i.e. the model's predictive power), the difference will be small.

$\endgroup$
3
$\begingroup$

Purely in terms of overfitting, and assuming you train both for equal amounts of time, 70/30 is probably better but performance is not going to be very good. Not training on %30 of data will make both training and test results equally bad (in my opinion). But it won't overfit, that is for sure. Cross validation (you have in mind 90/10, I assume) will take a long time, so that won't have enough time to train and it might be overfitting more compared to 70/30, but as it is going to see all training samples %90 at a time, there is a good chance it will train better. So, at the end of the day, it will overfit more but perform better.

If you are asking which is better overall, performance and overfitting, I say it depends on the size of your dataset. If you have millions of samples in it, you can even use a 98/1/1 for training, testing and validation and still be OK.

Edit: Thinking a little more about it, even if the time is not an issue the situation will roughly be the same. But you will know the performance of the model on new data to a higher certainty with cross validation.

$\endgroup$
4
  • 1
    $\begingroup$ What do you mean with: ",it will overfit a more but perform better"? $\endgroup$ Nov 17 '19 at 11:14
  • 1
    $\begingroup$ Using loss as a criteria, I mean both training and test results will be higher, but the ratio of test loss to training loss will be lower compared to 70/30 split. $\endgroup$
    – serali
    Nov 17 '19 at 11:19
  • 1
    $\begingroup$ I meant both training and test results will be lower, not higher in above comment. And also training loss to test loss ratio. It is loss, not accuracy so the lower the better. I am unable to edit the comment, sorry for the second comment. $\endgroup$
    – serali
    Nov 17 '19 at 11:41
  • $\begingroup$ No i understood what you are trying to say, thank you $\endgroup$ Nov 17 '19 at 11:44
0
$\begingroup$

Both methods are fine if used properly. As a rule of thumb, when training time is not an issue, use split method if you have more data than you can use in your model and cross-validation if not. I would suggest handling overfitting by some other means.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.