Validation accuracy less than training accuracy (with no sigh of overtraining)

Question

I am working with a deep CNN with over 100k sample data. I divided it up into 75% training, 12.5% validation and 12.5% for testing. As I train my network, the training accuracy approaches near 100% accuracy. The validation accuracy approaches 70-90% accuracy. The validation accuracy is always increasing and never decreases so I do not believe that the network is over-training.

The training accuracy is similar to the validation accuracy but both are less than the training accuracy.

My question is, what is causing my validation/training data to trail the training data? Is it because my validation/training sets contains sample types which are not found in the training set? What else might be causing this?

Additionally, between epochs, I see this 'stair case' in learning in that I see a huge jump in accuracy as soon as a new epoch starts. I am shuffling my data between epochs. What might be causing this jump in accuracy?

Also, if there are more technical terms for the events that I am describing please let me know so that I can further research these.

Thank you!

blue = training, black = validation

Answer 1

The validation trend doesn't inform you much about real overfitting, cause the model hyperparameters are optimized based on the validation set. Reason why usually the validation scores are better than the test ones. So to truly check overfitting you should constantly look at the test scores.

The jumps make me think that you're using all training instances each epoch, so no matter if you shuffle or not, the model performs better cause it starts going trough already seen examples.

In general, don't expect to reach a test accuracy similar to the training one, it happens only with perfect toy datasets. And if you have reasons to think the test accuracy should be much closer than what you are observing then inspect more closely your dataset and the splitting you're performing. Specifically check if you have imbalanced classes shown more in validation/test than training, larger variance in data features for validation/test than training or any other form of imbalance you might check depending on the type of data you're using.