2
$\begingroup$

I am new to deep learning.

I am training a model and I am getting a root mean squared error (RMSE) greater on the test dataset than on the training dataset.

enter image description here

What could be the reason behind this? Is this acceptable to get the RMSE greater in test data?

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ I think you should provide more details such as 1. your model architecture (I assume you are using a neural network) and 2. a plot of the evolution of the error and performance (on both the test and training data) throughout the epochs. $\endgroup$ – nbro♦ Mar 31 '20 at 13:19
1
$\begingroup$

It is common to have root mean squared error (RMSE) greater on the test dataset than on the training dataset (this is equal to having accuracy/score higher for model in training dataset than test dataset). This normally happens because the training data are assesed on the same data that have been learnt before, while the test dataset may have data that are unknown / not common that may give more errors or misclassification when doing prediction.

But if your model shows your test dataset have way too high RMSE result rather than your training dataset RMSE result, it may indicates that overfitting happens.

If overfitting happens, there are a lot of reasons this could happen. Referenced from https://elitedatascience.com/overfitting-in-machine-learning, some factors that causes overfitting are:

  • Complexity of data (e.g. there are irrelevant input features). This can be solved with removing irrelevant input features.
  • Not enough training data. This can be solved by training with more data (Eventhough this may not always succeed. Sometimes it may give noise towards data), etc.
Improve this answer
$\endgroup$
0
$\begingroup$

RMSE stands for Root Mean Squared Error. As the name suggests, it is calculated by taking the square root over the mean of the squared errors of individual points.

It is normal for the test error to be higher than the train error and in most cases, the test error will be greater than the train error.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ This answer doesn't answer OPs question at all. Please clarify why test error will be more. And how much error margin between test and train is acceptable. $\endgroup$ – user9947 Mar 30 '20 at 21:42
0
$\begingroup$

I am training a model and i am getting test results greater than train results.

You don't give us too many details, but most probably it's underfitting.

What could be the reason behind this?

  • Underfitting is often a result of an excessively simple model.

  • Too much regularization techniques were used.

Is this acceptable to get the RMSE greater in test data?

Yes, but that indicates a problem with model, so you should be aware of the consequences.

Improve this answer
$\endgroup$
8
  • $\begingroup$ Given the actual question, I am pretty sure that by "results" the OP meant "error" and not "performance" (e.g. accuracy). $\endgroup$ – nbro♦ Mar 31 '20 at 12:52
  • $\begingroup$ Your comment isn't helpful, sorry. $\endgroup$ – fuwiak Mar 31 '20 at 12:53
  • $\begingroup$ You answered by thinking that the OP got higher performance on the test data, but the OP got a higher error on the test data, not higher performance. Is this clearer now? $\endgroup$ – nbro♦ Mar 31 '20 at 12:56
  • $\begingroup$ Where I said about performance, uh? $\endgroup$ – fuwiak Mar 31 '20 at 12:57
  • $\begingroup$ Nowhere, but having a higher test error is not necessarily a sign of underfitting, although you say "probably". $\endgroup$ – nbro♦ Mar 31 '20 at 13:00
0
$\begingroup$

The training error (on any error metrics, not only for RMSE) will usually be less than the test error because the same data used to fit the model is employed to assess its training error. In other words, a fitted model usually adapts to the training data and hence its training error will be overly optimistic (too small). In fact, it is often the case that the training error steadily decreases as the size of the model increases.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.