I understand that RMSE is just the square root of MSE. Generally, as far as I have seen, people seem to use MSE as a loss function and RMSE for evaluation purposes, since it exactly gives you the error as a distance in the Euclidean space.

What could be a major difference between using MSE and RMSE when used as loss functions for training?

I'm curious because good frameworks like PyTorch, Keras, etc. don't provide RMSE loss functions out of the box. Is it some kind of standard convention? If so, why?

Also, I'm aware of the difference that MSE magnifies the errors with magnitude>1 and shrinks the errors with magnitude<1 (on a quadratic scale), which RMSE doesn't do.

  • $\begingroup$ 1.) Ease of derivative. 2.) Don't have to worry about ~0 in denominator causing huge gradient 3.) But to me the most important is mathematical convenience, someone might easily make the mistake of RMSE is just equal the difference $y-y'$ instead of root of mean square of $y-y'$. The answer this might start depending on conventions. 4.) In maths (Don't know the reason and might be inaccurate) we mainly work with variances instead of standard deviation. $\endgroup$ – DuttaA Aug 31 '19 at 17:37
  • $\begingroup$ Thanks @DuttaA , I think the comment you've given is quite good, that it can be one of the answers to this question. So, please post it as an answer below :) $\endgroup$ – Gokul NC Sep 1 '19 at 5:10
  • $\begingroup$ @nbro I thought that having the question title as "RMSE vs MSE" would be more SEO optimized since that's how most of the people search, anyway thanks for the edit. :) $\endgroup$ – Gokul NC Sep 1 '19 at 5:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.