How to handle an unbalanced dataset when training object detection algorithms?

Question

I am training an object detection model, and I have some very highly unbalanced data annotations. I have almost 11,000 images, all with dimensions of 1024 $\times$ 1024. Within those images I have the following number of annotations:

*Class 1 - 40,000
*Class 2 - 25,000
*Class 3 - 900
*Class 4 - 500

This goes on for a few more classes.

As this is an object detection algorithm that was annotated with the annotation tool Label-img, there are often multiple annotations on each photo. Do any of you have any recommendations as to how to handle fine-tuning an object-detection algorithm on an unbalanced dataset? Currently, collecting more imagery is not an option. I would augment the images and re-label, but since there are multiple annotations on the images, I would be increasing the number of annotations for the larger classes as well.

Note: I'm using the Tensorflow Object Detection API and have downloaded the models and .config files from the Tensorflow 2 Detection Model Zoo.

Answer 1

One thing to try first is Focal Loss. This particular loss works well for classification or object detection where your dataset is unbalanced and contains many classes. In short, the loss suppresses highly confident predictions and gives the model more room to learn from other less confident classes.

You can read this blog to have more intuition about focal loss.