0
$\begingroup$

enter image description here

For the UNSW-NB15 dataset i receive spikes in the loss function during training. The algorithms see part of this UNSW dataset a single time. Loss function is plotted after every batch. For other datasets I don't experience this problem. I've tried different optimizers and loss functions but this problem remains with this dataset.

I'm using the .fit_generator() function from keras. Is there anyone experience this problem using keras with this function ?

thanks in advance.

$\endgroup$
0
$\begingroup$

The spikes could be caused by many reasons: insufficient model capacity, incorrect label, buggy input parsing, ... Finding out the culprit requires some detective work. For instance, you could apply the learned model to the whole train set and manually examine the datapoints which result in the highest loss. Alternatively, you could compare the learning outcomes of different models (both weaker and stronger).

$\endgroup$
0
$\begingroup$

Andrew Ng explains with great details in Deep Learning Course appears in the image below. He also focuses on some corners can cause this problem :

  1. some mislabeled examples in the dataset.
  2. the size of your mini batch or change it with GD.

enter image description here

$\endgroup$
  • $\begingroup$ This is not the same as this case, the mean of spikes in Andrew Ng's picture is pretty much a smooth curve, whereas the mean in OP's graph it is irregular. $\endgroup$ – DuttaA Mar 23 at 19:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.