I'm training an object detection model (SSD300) to detect and classify body poses in thermal images.
Even I have more than 2k different poses, but the background does not change much (I have only 5 different points of view).
I trained my model on these images (70% for the training and 30% for validation).
Now, I want to evaluate the model on an unbiased dataset.
Should I keep images of my dataset for this purpose or should I use a real life dataset ?
(A good solution would be to have a real life training set, but I don't have)
I tried both, but as expected, I have an mAP=0.9 when evaluated on similar pictures and mAP=0.5 when evaluated on completely different images.
Bonus question: is mAP a relevant metric when I want to show result to a client ? (e.g a client doesn't understand if I tell him "my model has a mAP=0.7")
Precision-Recall ? (but I have to choose a pose classification threshold...)