1
$\begingroup$

I am new to deep learning.

I am training a model and I am getting a root mean squared error (RMSE) greater on the test dataset than on the training dataset.

enter image description here

What could be the reason behind this? Is this acceptable to get the RMSE greater in test data?

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ I think you should provide more details such as 1. your model architecture (I assume you are using a neural network) and 2. a plot of the evolution of the error and performance (on both the test and training data) throughout the epochs. $\endgroup$
    – nbro
    Mar 31, 2020 at 13:19

4 Answers 4

Reset to default
1
$\begingroup$

It is common to have root mean squared error (RMSE) greater on the test dataset than on the training dataset (this is equal to having accuracy/score higher for model in training dataset than test dataset). This normally happens because the training data are assesed on the same data that have been learnt before, while the test dataset may have data that are unknown / not common that may give more errors or misclassification when doing prediction.

But if your model shows your test dataset have way too high RMSE result rather than your training dataset RMSE result, it may indicates that overfitting happens.

If overfitting happens, there are a lot of reasons this could happen. Referenced from https://elitedatascience.com/overfitting-in-machine-learning, some factors that causes overfitting are:

  • Complexity of data (e.g. there are irrelevant input features). This can be solved with removing irrelevant input features.
  • Not enough training data. This can be solved by training with more data (Eventhough this may not always succeed. Sometimes it may give noise towards data), etc.
Improve this answer
$\endgroup$
1
  • $\begingroup$ I concur with the comment from @Angela Marpaung. You will always are going to have a higher RMSE in testing than training because testing hasn't been seen by the model. Remember models tend to memorize the answer so showing new data to the model makes them struggle to find the answer in the figurative sense. If you have a way disparate higher value of RMSE in testing that may indicate overfitting, but looking at the scale of your values in training/testing for RMSE it looks like the model can generalize. In this situation your model is a generalized model. Hope this helps. $\endgroup$
    – user_dhrn
    Mar 21 at 4:00
0
$\begingroup$

RMSE stands for Root Mean Squared Error. As the name suggests, it is calculated by taking the square root over the mean of the squared errors of individual points.

It is normal for the test error to be higher than the train error and in most cases, the test error will be greater than the train error.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ This answer doesn't answer OPs question at all. Please clarify why test error will be more. And how much error margin between test and train is acceptable. $\endgroup$
    – user9947
    Mar 30, 2020 at 21:42
0
$\begingroup$

I am training a model and i am getting test results greater than train results.

You don't give us too many details, but most probably it's underfitting.

What could be the reason behind this?

  • Underfitting is often a result of an excessively simple model.

  • Too much regularization techniques were used.

Is this acceptable to get the RMSE greater in test data?

Yes, but that indicates a problem with model, so you should be aware of the consequences.

Improve this answer
$\endgroup$
0
0
$\begingroup$

The training error (on any error metrics, not only for RMSE) will usually be less than the test error because the same data used to fit the model is employed to assess its training error. In other words, a fitted model usually adapts to the training data and hence its training error will be overly optimistic (too small). In fact, it is often the case that the training error steadily decreases as the size of the model increases.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .