I'm working on a regression problem with a CNN in which the input is a single image, and the output is an angle in degrees (which determines a specific measure related to the image).
Sometimes, the model fails to retrieve the output accurately (for output angles wider than 20°). By analyzing the data, I can suspect there is a problem of imbalance since there are a lot of training samples with outputs between -20° and 20° but very few for wider angles (the wider the angle, the fewer the examples).
There is no possibility to generate more data to balance the training set artificially. I want to try a robust loss function focusing more on wide angles.
My model is typically trained with MSE, but I would like to implement a loss function in pytorch, which is linear for ground-truth values lower than 20° (in absolute value) and becomes quadratic for greater values. This is similar to the Huber Loss, but 1) the conditions would be the opposite. 2) The condition of the Huber function would be based on the GT instead of the absolute difference between the GT and the output.
So, my question is whether this solution would make sense from an ML point of view since I haven't seen something like this in ML literature. Or is there maybe a more "standard" way to face this problem ? (apart of data augmentation)
Thanks in advance.