I am trying to understand the average precision (AP) metrics in evaluating the performance of deep-learning based object detection models. Suppose we have the following ground true (four objects highlighted by four blue arrows):
where we have labelled four objects:
person 25 16 38 56
person 129 123 41 62
kite 45 16 38 56
kite 169 123 41 62
And when feeding the above image to an object detector, it gives the following outputs:
It's easy to see that the object detector identified another object with low confidence:
person 0.4 25 16 38 56
person 0.2 129 123 41 62
kite 0.3 45 16 38 56
kite 0.5 169 123 41 62
kite 0.1 769 823 141 162 <-------- a "kite"
In my humble opinion, this is an erroneous behavior of the object detector, which should be counted as a "false positive".
However, since the "kite" has a quite low confidence score (0.1), when using the standard mAP algorithm to compute the performance, I got the following output (I am using code from here to compute the mAP):
AP: 100.00% (kite)
AP: 100.00% (person)
mAP: 100.00%
So here are my questions and confusions:
from what kind of design intension, the
APis designed in a way such that objects with low confidence score are ignored and therefore in this case we are with flying colors.Is there any metrics that can take this extra "kite" into consideration and therefore would output one "false positive" of the object detection model? I am just thinking that in this way, we can further proceed to improve the accuracy of this model during training.
