I have been training my CNN for a bit now and, while both the training loss and the training error curves are going down during training, both my validation loss and my validation error curves are kind of zig-zagging and oscillating along the epochs. What does this represent?

  • $\begingroup$ This is an old question, but if it would be great if you could provide a plot of the training and validation loss and error. Moreover, you should provide more context, such as provide the architecture of your CNN. If you're using TF, you could do plot_model(my_model). $\endgroup$
    – nbro
    Dec 14, 2020 at 10:30

1 Answer 1


Without more information the best diagnostic is:

You potentially have a bug in your code.


Neural network systems are capable of failing silently. The system can still appear to "learn" even in the presence of said bug - a while back I had a very similar issue with a toy CNN project. My network could get 99% accuracy on the training set but always achieved 8-13% on the validation set. This looked like overfit but none of the methods solved the issue. Finally, I found that I wasn't feeding the data correctly into the network during train time but I was feeding the data correctly during validation time.


Provide the following for better diagnostic:

  • Is this a custom CNN you hardcoded?
  • are you using python?
  • what's the objective function?
  • can you provide the loss curve for the training set?
  • show us some code too (this will be very helpful for working toward a solution)

I hope this helps, and best of luck!


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .