0
$\begingroup$

enter image description here

What does this difference in train and test accuracy mean?

$\endgroup$

2 Answers 2

0
$\begingroup$

For the Logistic Regression Classifier and SVC, the train_accuracy and test_accuracy are very similar; thus, there is no evidence for over or under fitting. However, the KNN, shows a train_accuracy that is lower than your test_accuracy, which suggests overfitting. Alternatively, depending on your sample size, it is entirely possible that this variation is within the expected range of error.

In machine learning, the model is trained on one dataset (i.e., the training data), but performing model evaluation (i.e., estimate "real world" performance by calculating accuracy or some other metric) on the same dataset that you trained on would be biased. Instead, the evaluation is performed on an independent identically distributed dataset (i.e. the test data) to get an unbiased estimate of model performance. However, it is still important to compare accuracy (or whatever performance metric you choose) between training and testing to assess for over- or underfitting.

Overfitting means that your model "fit" the training data very well and achieves high accuracy on that data. However, the model performs poorly (or less well) on the test data. This is because the model not only "fits" the signal in the training data but also the noise. The problem is that noise is random and it will surely be different in the test set. Therefore, the model doesn't generalize well to other datasets.

Underfitting means that the model performs poorly on both datasets. This is typically because the model does not have the "expressivity" or "capacity" to learn the signals within the data or that the data is random and has no "signal" to speak of (and, therefore, nothing from which to learn).

$\endgroup$
-1
$\begingroup$

Actually, in the strict sense, you cannot decide overfitting here, because overfitting is tested using losses, not metrics like accuracy.

$\endgroup$
4
  • $\begingroup$ For the downvote, why? This is correct :) $\endgroup$
    – Dr. Snoopy
    Commented Nov 14, 2022 at 12:23
  • $\begingroup$ To my knowledge overfitting is a phenomenon that's unbound to any metric - it only means that your model is overly specific to the training data. You can identify an overfitted model with accuracy as well as with loss, no? $\endgroup$
    – Chillston
    Commented Dec 5, 2022 at 15:00
  • $\begingroup$ @Chillston no, the definition is to only use loss (since that is what the model is being trained on), not any metric, because metrics measure different things. $\endgroup$
    – Dr. Snoopy
    Commented Dec 5, 2022 at 15:46
  • $\begingroup$ Interesting, do you have an intuition why this is defined so strictly? Where is the definition from? Not trying to be cheeky - just interested because its not super intuitive to me. $\endgroup$
    – Chillston
    Commented Dec 5, 2022 at 17:46

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .