Questions tagged [cross-validation]

For questions related to the cross-validation technique used e.g. in machine learning or statistics.

8 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
13
votes
1answer
164 views

Will parameter sweeping on one split of data followed by cross validation discover the right hyperparameters?

Let's call our dataset splits train/test/evaluate. We're in a situation where we require months of data. So we prefer to use the evaluation dataset as infrequently as possible to avoid polluting our ...
1
vote
0answers
53 views

How can I split the data into training and validation sets such that entries with a certain value are kept together?

I have the following kind of data frame. These are just example: A 1 Normal A 2 Normal A 3 Stress B 1 Normal B 2 Stress B 3 Stress C 1 Normal C 2 Normal C 3 Normal ...
0
votes
1answer
108 views

How to fill NaNs in Cross-Validation?

I have been searching this but did not find the answer, so sorry if this is a duplicated question. I was working with cross-validation, where some doubts came to my mind, and I am not sure about which ...
0
votes
0answers
15 views

Why does 0.8:0.2 divided dataset have a much greater AUROC than its 5-fold cross validated counterpart's mean AUROC?

I trained a dataset with a 5-fold cross validation to search for hyper-parameters by an AUROC metric. For splitting I used ...
0
votes
0answers
18 views

Is it a fair evaluation if each model of k-fold cross validation is trained with different epochs and the mean AUC score gathered out of k folds?

Let's say I have a dataset for binary classification. And I am going to conduct 5-fold cross-validation and get AUC scores for each fold (mean AUC score too). However, if I set the training epoch to $...
0
votes
2answers
126 views

How to avoid over-fitting using early stopping when using R cross validation package caret

I have a data set with 36 rows and 9 columns. I am trying to make a model to predict the 9th column I have tried modeling the data using a range of models using caret to perform cross-validation and ...
0
votes
0answers
29 views

What would happen if validation set was the same as the training set?

Just to check if everything is working properly in my neural net, I set my validation data to be the same as my training data, expecting to achieve a better NRMS for validation data (since it uses ...
0
votes
0answers
23 views

Different result from k-cross validation model and Train-Validation-Test split model ? (AI fresher question)

I am starting to learn about Neural Network and I have come into one problem that I am still learning how to figure it out. I have a dataset with shape (105,96) (105 samples and 95 first column as ...