2
$\begingroup$

I learned that when creating neural networks the go to was to overfit and then to regularize. However I am now in a situation where, when I make the model more complex (more layers, more filters, ...) my scores become worse.

I am training a CNN to predict pollution 6 hours in advance. The input I give to my model is the pollution of the past 18 hours.

Can I safely say that because there probably is a lot of noise in this data that, that is the reason when increasing my complexity, my model becomes worse?

Improve this question
$\endgroup$
4
  • $\begingroup$ It is mostly likely due to overfitting. What's your training loss is like? Also, do you also increase regularization effects, e.g., dropout rate, weight-decay's delta, as you make your model more complex? $\endgroup$
    – SpiderRico
    Mar 18, 2020 at 17:28
  • $\begingroup$ Yes it is definetly due to overfitting, but I assume that it is overfitting on noisy data, which makes it even worse. It gives quite good results on the training data, but very bad on the validation data. Even after regularizing the hell out of it, results do not become better than just a simple model with only 2 layers (in comparison to 4). $\endgroup$
    – Tibo Geysen
    Mar 18, 2020 at 18:59
  • $\begingroup$ I see. I mean if the quality of your data is low then I dont think your validation accuracy can increase no matter what you do. But then I might be wrong. $\endgroup$
    – SpiderRico
    Mar 18, 2020 at 19:03
  • $\begingroup$ Generally for your situation I think that the noise factor in the data is keeping the validation accuracy low. Another possibility is that you do not have a sufficient data set size to fully capture the probability distribution of the data. How many training samples do you have? Also how did you select the validation data, It should be a random sample of the data set so its distribution matches that of the training set. $\endgroup$
    – Gerry P
    Mar 19, 2020 at 4:31

0

Reset to default

You must log in to answer this question.

Browse other questions tagged .