I am working with a deep CNN with over 100k sample data. I divided it up into 75% training, 12.5% validation and 12.5% for testing. As I train my network, the training accuracy approaches near 100% accuracy. The validation accuracy approaches 70-90% accuracy. The validation accuracy is always increasing and never decreases so I do not believe that the network is over-training.
The training accuracy is similar to the validation accuracy but both are less than the training accuracy.
My question is, what is causing my validation/training data to trail the training data? Is it because my validation/training sets contains sample types which are not found in the training set? What else might be causing this?
Additionally, between epochs, I see this 'stair case' in learning in that I see a huge jump in accuracy as soon as a new epoch starts. I am shuffling my data between epochs. What might be causing this jump in accuracy?
Also, if there are more technical terms for the events that I am describing please let me know so that I can further research these.
blue = training, black = validation