Let's say I have a dataset for binary classification. And I am going to conduct 5-fold cross-validation and get AUC scores for each fold (mean AUC score too). However, if I set the training epoch to $30$, and, with that classifier, test all my test folds iteratively, I will be getting AUC scores of test folds at specific weights of a classifier (at $30$s epochs).

Is it a fair evaluation if I watch some metrics (like validation loss) until stagnation and reduce the learning rate accordingly for each fold separately and get their respective AUC score, then report the mean AUC score?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.