I am training an object detection machine learning pipeline. Among the many metrics provided out of the box by tensorflow object detection API, I look at total_loss and DetectionBoxes_Precision/mAP@.75IOU:
Here the x-axis is the number of steps i.e. model experience. The orange line is for the training data and the blue line is for the validation data.
From the loss graph I would conclude, that at approx 2k steps overfitting starts, so using the model at approx 2k steps would be the best choice. But looking at the precision graph, training e.g. until 24k steps would be a much better model. Which one is the best model?
Here, loss and the precision metric where picked just for illustrating the dilemma, there are many more metrics available, leading to multiple conclusions about when overfitting actually starts.