0
$\begingroup$

My project include classification of images into several classes.

I'm having a strange issue related to adding mixup augmentation. The accuracy of the training set and the validation set keep rising during all training.

However, the accuracy on the test set improves significantly until epoch 80 (over baseline without mixup - grey), but when the learning rate is reduced it collapses. Note that each 40 epochs I reduce the learning rate.

Is it learning rate/optimizer/scheduler problem? If so..what should I do? I already tried cosine schedulers and linear schedulers.

enter image description here

Improve this question
$\endgroup$
5
  • $\begingroup$ what dataset are you using/how large is it? Could be simply over fitting from these graphs. $\endgroup$
    – Edoardo Guerriero
    Commented Jan 25, 2022 at 12:09
  • $\begingroup$ As you can see these are validation and test graph not training graphs. Validation keep increases. Also, the grey in the test set is also increasing. Only the green (with mixup) is decreasing. The question is how to solve it $\endgroup$
    – BestR
    Commented Jan 25, 2022 at 13:42
  • $\begingroup$ Your still optimizing on your validation set, so that doesn't prove it's not over fitting. And as you can see by decreasing learning rate every 40 epochs you keep fine tuning on the same batches again and again, improving slightly (cause after a while the weights basically stop changing) without any benefit on the test set, which remain slightly the same (if you smooth the scores with moving average it's most likely flat, not decreasing). Plus you have a 10 to 20% difference between the 2 sets scores, which is another strong indicator for over fitting. $\endgroup$
    – Edoardo Guerriero
    Commented Jan 25, 2022 at 14:14
  • $\begingroup$ You are right, in the grey graph the validation is basically flat because lr is low. if i dont change it varies again. But also in low lr, it improves a tiny bit. You think about early stopping? And the differences between the sets is becuase they are from different distributions. $\endgroup$
    – BestR
    Commented Jan 25, 2022 at 14:44
  • $\begingroup$ well if your train, test and validation sets are different that's the very first thing to address. They are suppose to be representative of the same distribution, if not the test results are useless and not even to be interpret at all (otherwise it's like comparing apples and bananas and wondering why they don't look the same). $\endgroup$
    – Edoardo Guerriero
    Commented Jan 25, 2022 at 16:07

0

Reset to default

You must log in to answer this question.