0
$\begingroup$

I am looking for some advice regarding the best choice of binary classification model based on training, validation and test set results. Model 1 (results in 1st image) shows better test set results than Model 2, but Model 2 (results in 2nd image) shows results that seem more intuitive to me with better training set performance than its test set performance. I feel as if the Model 1 test set results might have been a bit of a fluke, whereas Model 2 appears more like a well trained model with more long-term reliability.

Any advice on this is much appreciated.

Model 1 Results

Model 2 Results

$\endgroup$

1 Answer 1

0
$\begingroup$

Generally if a model A performs worse than model B on "Test set" while performing better on "Train/Val set" then model A has suffered from overfitting.

What is overfitting?

So, in your case Model 1 should give you long term reliability against unseen data.

$\endgroup$
2
  • $\begingroup$ Thank you very much for your answer Marib. $\endgroup$ Commented Aug 8, 2022 at 12:46
  • $\begingroup$ If you are satisfied with the answer, can you mark it as accepted $\endgroup$ Commented Aug 8, 2022 at 13:07

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .