Object detection uses mAP as the metrics. But if we are only interested in classification once the object in a bounding box is extracted, what metrics should we use? Thanks!
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
1
For classification problems, using crossentropy and bce is always my go-to option. It usually performs best for these types of problems. However, using MSE or its variants is also common in some cases. Hinge loss is another one that you might want to look at if the previous options didn't work.
-
$\begingroup$ Your answer could be improved with additional supporting information. Please edit to add further details, such as citations or documentation, so that others can confirm that your answer is correct. You can find more information on how to write good answers in the help center. $\endgroup$– Community BotAug 30, 2022 at 1:53
$\begingroup$
$\endgroup$
You could use top-n accuracy with a couple different n values like is normally done in simple classification tasks.
For a given class prediction, you score it as 1 if the ground truth label is in the top $n$ prediction probabilities, and 0 if it isn't in the top $n$. Then you average all these values over all your relevant predictions. (see https://stats.stackexchange.com/questions/95391/what-is-the-definition-of-top-n-accuracy for more on this)
For object detection specifically you could only count boxes towards your accuracy score that have sufficient IOU with a ground truth box so that it is only evaluating class accuracy on accurate bounding boxes.
You can also make a confusion matrix (https://en.wikipedia.org/wiki/Confusion_matrix) to see which classes are being commonly confused.
The problem with using class loss as your metric, as suggested in one of the other answers, is that it's one of the parameters you'd want to play with while aiming for a better accuracy so you wouldn't have a consistent way to score your experiments.
