Skip to main content
1 of 3
hanugm
  • 4k
  • 3
  • 28
  • 59

What are the 'noisy factors' that leads to overfitting?

Consider the following excerpt on overfitting from p13 of Linear Regression.

There is a problem with learning weights that make the model perfectly match the training data. If a feature is perfectly predictive of the outcome because it happens to only occur in one class, it will be assigned a very high weight. The weights for features will attempt to perfectly fit details of the training set, in fact too perfectly, modeling noisy factors that just accidentally correlate with the class. This problem is called overfitting.

What are the 'noisy factors' here? Does it refers to the features that are irrelevant to the class label?

Or does it mean the noise/errors in the values taken by features that accidentally correlate with the class label?

hanugm
  • 4k
  • 3
  • 28
  • 59