I'm asking because classification problems have very concrete metrics like accuracy that are totally transparent to understand.

Whereas regression models seem to have a very large number of possible evaluation strategies and to me at least it is not clear which (if any) of them is as reliable/interpretable as accuracy is in classification problems.

Possible Candidates:

  • Regular loss (e.g MAE): MAE is potentially quite interpretable, but again interpretation depends upon distribution statistics which vary across regression problems.
  • MAPE/Relative Loss: This is interesting and is potentially decently similar to accuracy. Yet it has obvious draw backs, like the true value being extremely small causing explosion of loss values & there being no incorporation of overall distribution statistics for the output values.
  • Chi-squared test: I like the idea of this but I have not seen it used at all for NN regression for some reason. I'm not sure why and I'm curious if people think it would be a good idea to use it for that.
  • (adjusted) R^2 coefficient: Another statistic that seems great in theory, but again I see almost never being used for NNs and I'm not sure why. This has the great advantage of being a 'bounded'/'normalized' metric like accuracy and in theory is should be just as interpretable. Why is it not used for NNs?

I've gotten no answer on this, but after some reflection I've come to accept R^2 as regression's analogue to accuracy. I have no idea why it is not used by more people in deep learning, but I recently started using it and it's exactly what I was hoping for! It scales from 0 to 1 and tells you in no uncertain terms how useful the model is and when it becomes unrealistic to improve the model further (e.g. at R^2=0.99). Additionally, R^2=0 when the model starts guessing only the mean of the data, which means it immediately reveals an important and recurring issue in regression which can otherwise go undetected for a while.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.