A model can be classified as parametric or non-parametric. How are models classified as parametric and non-parametric models? What is the difference between the two approaches?
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
Parametric Methods
A parametric approach (Regression, Linear Support Vector Machines) has a fixed number of parameters and it makes a lot of assumptions about the data. This is because they are used for known data distributions, i.e., it makes a lot of presumptions about the data.
Non-Parametric Methods
A non-parametric approach (k-Nearest Neighbours, Decision Trees) has a flexible number of parameters, there are no presumptions about the data distribution. The model tries to "explore" the distribution and thus has a flexible number of parameters.
Comparision
Comparatively speaking, parametric approaches are computationally faster and have more statistical power when compared to non-parametric methods.
answered Sep 27, 2020 at 2:27
$\begingroup$
$\endgroup$
I provided some details but the most important excerpt is from Stuart Russell and Peter Norvig's AIMA book:
A learning model that summarizes data with a set of parameters of fixed size (independent of the number of training examples) is called a parametric model. No matter how much data you throw at a parametric model, it won’t change its mind about how many parameters it needs.
For nonparametric models ask yourself a question: What is the number of parameters of the decision tree?
As the decision tree is an example of nonparametric model the number of parameters in a decision tree depends on the quantity of the data. The more data we have means more parameters.
