I have been training my CNN for a bit now and, while both the training loss and the training error curves are going down during training, both my validation loss and my validation error curves are kind of zig-zagging and oscillating along the epochs. What does this represent?
Without more information the best diagnostic is:
You potentially have a bug in your code.
Neural network systems are capable of failing silently. The system can still appear to "learn" even in the presence of said bug - a while back I had a very similar issue with a toy CNN project. My network could get 99% accuracy on the training set but always achieved 8-13% on the validation set. This looked like overfit but none of the methods solved the issue. Finally, I found that I wasn't feeding the data correctly into the network during train time but I was feeding the data correctly during validation time.
Provide the following for better diagnostic:
- Is this a custom CNN you hardcoded?
- are you using python?
- what's the objective function?
- can you provide the loss curve for the training set?
- show us some code too (this will be very helpful for working toward a solution)
I hope this helps, and best of luck!