During training of my models I often encounter the following situation about training (green) and validation (gray) loss:

training and validation loss

Initially, the validation loss is significantly lower than the training loss. How is this possible? Does this tell me anything important about my data or model?

One explanation might be, that training and validation data are not properly split, i.e. the validation might primarily contain data, that the model can easily represent. But then why do the curves cross after epoch 30? If this is because of overfitting, then I would expect the validation loss to increase, but so far both losses are (slowly) decreasing.

There is a related question at Data Science SE, but it doesn't give a clear answer.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.