0
$\begingroup$

I working on a classification problem that needs to detect patterns on a time serie. Basically, there's a catch-all class that means "no pattern detected", the other are for the specific patterns. The data is imbalanced (ratio 1/10 at least), but I adapted the class weights.

I'm able to overfit successfully on a few days of data, but when I train on 2 years of data, the model seems stuck on class1 "no pattern detected" for a veeeery long time. I've tried several learning rates, but it doesn't make the convergence happen significatively faster.

Is it a better starting point for my training to use the overfitting model's weight as a starting point? Could this allow the model to converge faster?

$\endgroup$
0
$\begingroup$

The first thing which you first have to understand is that does your trained model is working efficiently with both the training and testing data. If yes then its not overfitting. There is only one case where over-fitting doesn't matter and that is when your testing data is same as training data and dealing with is very important or it could be disastrous. Therefore, to avoid overfitting you can try several methods and they are followings:

  1. Optimization Algorithms like sgd, Adam, RMSprop, Adagard, are few gradient descents.
  2. Loss function like Hinge loss could be useful.
  3. Parameter initialization greatly influence the optimization process.

Overall, Large datasets are better for building ML models but there are some methods there is an interesting article that i would recommend you to read: Article Link

I Hope this Helps!!!

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thx for your answer, but I think that you didn't understand the question. My problem here is not to avoid overfitting, I'm searching for a way to make my model train faster. $\endgroup$ – Sebastien Apr 30 at 16:58
  • $\begingroup$ @Sebastien If you are really looking for a faster training model you have to include the coding, datasets, and modules. Looking forward to help! $\endgroup$ – techPirate99 Apr 30 at 17:01
  • $\begingroup$ I'm not willing to include everything (this may become more complicated than my simple question. I just want to know if sometime, doing 1) train model on small data until overfit then 2) train model on all data can be faster than just 2) $\endgroup$ – Sebastien Apr 30 at 17:06
  • $\begingroup$ @Sebastien Droping small pieces of data until overfitting will converge faster in training data. And you really focusing on convergence then check article too link $\endgroup$ – techPirate99 Apr 30 at 17:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.