0
$\begingroup$

I have built a network that performans pretty well on my data. The issue I have is that for a larger number of epochs at the start of the training process the val/train acc/loss are stagnating (for some train/val splits the val_losses increase for a certain number of epochs) and then dropping dramatically.

What can be the reason for this, and how can I make it drop faster?

blue = val, red = train

Improve this question
$\endgroup$
4
  • 2
    $\begingroup$ You should add more info about the optimizer you're using, the hyper parameters (especially learning rate), if you're using also a scheduler and the loss you're optimizing. $\endgroup$
    – Edoardo Guerriero
    Nov 7, 2022 at 14:22
  • 1
    $\begingroup$ I agree with @Edoardo Guerriero. More training details are needed to help diagnose this issue. Some details about the data, task, and model would also be helpful. Finally, even though we can surmise from your text which curves are validation curves and which are training curves, a legend should always be included with your plots when plotting multiple curves as a matter of good practice and professionalism. This is an interesting problem. Help us help you. $\endgroup$
    – Snehal Patel
    Nov 8, 2022 at 13:22
  • $\begingroup$ @EdoardoGuerriero Yeah I know. I was more searching for a general overview. I do not think it's realistic to see my problem "solved" here (because the causes may be very varied). Eventually, I was looking for a starting point where I can research by myself to find out the cause.A list with most common reasons for such behavior. $\endgroup$
    – Skobo Do
    Nov 8, 2022 at 14:26
  • $\begingroup$ Upvoted this is a genuine question. Quite essential generic research topic too. $\endgroup$
    – Peter
    Nov 12, 2022 at 21:14

0

Reset to default

You must log in to answer this question.