I'm trying to work out an approach to balancing my dataset, which is a subset of a google openimages - some classes are represented orders of magnitude more than others and I am hesistant to simply throw away data. The approach I want to try next is roughly:
Add a single instance of every image to the training set. Crop out instances of the majority class from entire dataset and then add these cropped images to the dataset. keep adding these cropped images to training set until number of instances of 2nd majority class is close to number of instances of 1st majority class. Crop out instances of second majority class and repeat (2) keep going until all images have been cropped down to the minority class and dataset is roughly balanced.
This approach would mean that no instance of any class would be repeated more often than any other instance of the same class. My concern is that the minority classes will end up being predominantly represented in training by heavily cropped images. I'm not sure but I don't think this would matter in a two-stage object detector but I'm concerned that it might be a problem for a one-stage detector if, for some classes, it was predominantly trained on images that were cropped to various degrees.
The alternative to doing this would be undersampling ie. just throwing away most of the data which I'm hesistant to do.
Anyway, it would be really great to get some opinions on this in the context of Yolov3? Is yolov3 sufficiently robust at scale-invariance that training on many examples that are cropped/larger than what they are likely to be at test time?
(I'm fairly commited to some sort of oversampling because of the results in this paper which found best results generally from oversampling with CNN classifiers as opposed to undersampling, threshold adjustment etc. https://arxiv.org/pdf/1710.05381.pdf)