0
$\begingroup$

I am running a model with fixed hyperparameters. To my surprise/shock, the model converged extremely fast with the least loss possible.

I want to know the causes of this phenomenon. I have the following guesses:

  1. Underlying mapping is so simple.

  2. Hyperparameters are apt.

  3. Both.

Are there any other reasons for this phenomenon?

Improve this question
$\endgroup$
2
  • $\begingroup$ What you mean by "Hyperparameters are apt." ? $\endgroup$
    – pasaba por aqui
    Commented Feb 19, 2022 at 17:22
  • $\begingroup$ @pasabaporaqui I mean, correct hyperparameters for the model $\endgroup$
    – hanugm
    Commented Feb 20, 2022 at 8:31

3 Answers 3

Reset to default
1
$\begingroup$

send us your loss function plot over epochs ( or steps ). this will help to get a better guidance(use log scale for loss axis). sending more details of your learning process may help too.

but in this situation, i think you should decrease the learning rate and using the learning rate decrease method. this method helps you to see stepwise decrease of loss to the best losses. you can see more details of this method in this link.

Improve this answer
$\endgroup$
0
$\begingroup$

There might be several reasons for that:

  • the data is easily understood by the model you are using
  • the model you use is fitted to the problem
  • the problem complexity is low

There are a lot more reasons to explain the convergence of an algorithm.

Improve this answer
$\endgroup$
0
$\begingroup$

If a NN converges in a few steps to the absolute minimum of the loss function it means the loss function has a gradient (in the domain that the inputs defines) very regular, pointing to the absolute minimum.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .