In machine learning, there are several metrics to assess the quality of the models: accuracy, precision, recall, f measure, ROC (AUC), etc. There are cases when certain metrics are more appropriate than others. For example, accuracy is not a good metric when there is class imbalance in the dataset (although there are ways of attenuate this issue, e.g. resampling). ROC curves are typically used in binary classifications problems, even though they can also be used in multi-class classification problems (with some tricks). The confusion matrix is also a nice way to visualise the performance of the model (in the case of multi-class classification).

I am looking for an as much as possible comprehensive (which can also be concise) explanation of when a metric (at least, the ones mentioned above) can or should (or not) be used, depending on the dataset (balanced or unbalanced), type of problem (classification or regression), type of classification (binary or multi-class), etc. You can eventually add more details and metrics.

  • $\begingroup$ One more criteria for selecting a particular metric that could be added is the utility function of the end user. $\endgroup$ – naive Aug 10 at 14:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.