I have downloaded a pre-trained EfficientDet D2 model (Tensorflow 2.0) and trained it on some data (about 20000 images with 20 classes). I set the number of steps to 25000 and batch size to 3 (computer resources are not the best).
However, if I try to make predictions, the pre-trained model makes better predictions than the model I have trained on the additional data. Is this expected behaviour?
For example, an image of a person may be 78% accurate on the pre-trained model and only 54% accurate on the same image when trained.