I am working on a project where I am working on the Flickr-47 dataset to do logo detection and classification. My approach is to first finetune a YOLO v5 model with high recall to detect as many "logos" as it possibly can and then to classify the detected regions using some sort of feature extraction.
Here is an example of the actual image and what the model detects as logos.
Ground Truth Annotations:
My question is because there are instances as can be seen above where the number of regions detected by the model can be less or more than the actual regions to be detected in the ground truth image, how do I decide which ones to keep for classification?
P.S. I know I can apply non-max suppression to remove overlaps but the above can still persist if the YOLO detector finds something worth a logo, but isn't. I can also modify the IoU threshold and confidence threshold during detection inference, but I want to be able to extract as much as the YOLO model detects, and keep the filtering at a later stage.
Any ideas?