1
$\begingroup$

Below you have the plots of the training and validation errors for two different models. Both plots show the RMSE values for the validation dataset versus the number of training epochs. It is observed that models get lower RMSE value as training progresses.

The model associated with the first plot is performing quite well. The gap is quite narrowed.

First Plot

I think the model associated with this second plot is doing pretty good, but not as well as the other. The gap is much broader.

Second Plot

The model of the first plot was trained using a data set containing 1 million of ratings, while the second one used only 100K. I'm implementing the collaborative filtering (CF) algorithm. I am optimising it using SGD.

enter image description here

Are any of these models overfitting or underfitting?

Improve this question
$\endgroup$
4
  • $\begingroup$ I think you should add more relevant details to the question. Try to see from the viewpoint of an answerer and see what details you might need to answer this question (e.g model architecture, hyperparameters, etc) $\endgroup$
    – user9947
    May 16, 2019 at 5:11
  • 2
    $\begingroup$ What do you mean by "describe both plots in terms of model complexity"? What do you mean by "model complexity"? $\endgroup$
    – nbro
    May 16, 2019 at 13:59
  • $\begingroup$ if it is overfitting or underfitting $\endgroup$
    – NaveganTeX
    May 16, 2019 at 15:23
  • $\begingroup$ Regarding Model Complexity: (stats.ox.ac.uk/~sejdinov/teaching/sdmml15/materials/…) $\endgroup$
    – NaveganTeX
    May 17, 2019 at 2:47

2 Answers 2

Reset to default
1
$\begingroup$

I would say that your intuition is correct: the model associated with the first plot is likely to generalise more than the one associated with the second plot.

In both cases, it doesn't seem that your model has overfitted the training data. Overfitting often occurs when the training error keeps decreasing but the validation error starts to increase. In both your plots, both the training and validation errors keep decreasing (even if slowly, after a while).

Underfitting occurs when your model hasn't learned enough even about your training data. The smaller the training and validation error, the more likely your model has not underfitted, but the value of RMSE depends on the range of your inputs. See e.g. What are good RMSE values? for more info.

See also this article Overfitting and Underfitting With Machine Learning Algorithms for a general overview of the concepts of overfitting and underfitting.

Improve this answer
$\endgroup$
0
$\begingroup$

One possibility: If you are using a dropout regularization layer in your network, it is reasonable that the validation error is smaller than the training error. Because usually dropout is activated when training but deactivated when evaluating on the validation set. You get a more smooth (usually means better) function in the latter case.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .