I am trying to train a Unet network with Synthetic data to do binary segmentation due to the fact that is is not easy to collect real data.
And there is something in the training process that I do not understand.
I have a gap in the IoU metrics between the training and the validation (despite having really similar data).
My training Iou is around 95 % and my validation is around 70 %. And the dice loss is around 0.007. The IoU is calculated on the inverted mask used for the loss.
So I do not understand why there is this gap whereas the images in validation has been created from the same background dataset and the same object dataset which has been randomly placed on background ( + rotation and rescaled randomly). The only difference is an aggressive data augmentation used for training dataset.
In my opinion, it is not overfitting since the loss value and comportment is very similar for train and val. Moreover, it seems very unlikely to me that the model overfit with same backgrounds and objects or at least model should have very good IoU for train and val if it was overfitting.
So could the data augmentation lead to the model learning features which corresponds to data augmented data (even if loss is similar) and not to the real data explaining the gap in IoU between train and val ?