2
$\begingroup$

I am training an object detection machine learning pipeline. Among the many metrics provided out of the box by tensorflow object detection API, I look at total_loss and DetectionBoxes_Precision/mAP@.75IOU:

enter image description hereenter image description here

Here the x-axis is the number of steps i.e. model experience. The orange line is for the training data and the blue line is for the validation data.

From the loss graph I would conclude, that at approx 2k steps overfitting starts, so using the model at approx 2k steps would be the best choice. But looking at the precision graph, training e.g. until 24k steps would be a much better model. Which one is the best model?

Here, loss and the precision metric where picked just for illustrating the dilemma, there are many more metrics available, leading to multiple conclusions about when overfitting actually starts.

$\endgroup$
0
$\begingroup$

Generally, a model is considered to be overfitted when there's a huge gap between training and test/validation performances.

So during the training, you monitor the loss on validation data, and training data, and stop training if validation loss stagnates/increases given the training loss keeps decreasing.

In your scenario, I'm not sure what the total loss metric corresponds to. As I said, you've to measure loss on a held-out data other than training data to detect, and prevent overfitting.

$\endgroup$
1
  • $\begingroup$ Thank you @SpiderRico. According to your statement, I would stop training at around 2k steps. My question is: why should I stop training if validation based (held-out data) mean average precision (mAP) further increases? $\endgroup$ – user1091141 May 1 at 19:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.