6 votes

Should I choose a model with the smallest loss or highest accuracy?

You should choose the model A. The loss is just a differentiable proxy for accuracy. That said, the situation should be examined in more detail. If the higher loss is due to the data term, examine ...
user avatar
  • 469
4 votes
Accepted

Should I choose a model with the smallest loss or highest accuracy?

You should note that both your results are consistent with a "true" probability of 87% accuracy, and your measurement of a difference between these models is not statistically significant. With an 87% ...
user avatar
  • 23.8k
1 vote

how to decide the optimum model?

Testing each time on a test set is against the point of a train-val-test split. The reason test is important, is that you are only supposed to test on it when you think your model is good and ready ...
user avatar
  • 2,249
1 vote

Should I choose a model with the smallest loss or highest accuracy?

It depends on your application! Imagine a binary classifier that is always very "confident" - it always assigns P=100% to Class A and 0% to Class B, or vice versa (sometimes wrong, never uncertain!). ...
user avatar

Only top scored, non community-wiki answers of a minimum length are eligible