A model can be classified as parametric or non-parametric. How are models classified as parametric and non-parametric models? What is the difference between the two approaches?


2 Answers 2


Parametric Methods

A parametric approach (Regression, Linear Support Vector Machines) has a fixed number of parameters and it makes a lot of assumptions about the data. This is because they are used for known data distributions, i.e., it makes a lot of presumptions about the data.

Non-Parametric Methods

A non-parametric approach (k-Nearest Neighbours, Decision Trees) has a flexible number of parameters, there are no presumptions about the data distribution. The model tries to "explore" the distribution and thus has a flexible number of parameters.


Comparatively speaking, parametric approaches are computationally faster and have more statistical power when compared to non-parametric methods.


I provided some details but the most important excerpt is from Stuart Russell and Peter Norvig's AIMA book:

A learning model that summarizes data with a set of parameters of fixed size (independent of the number of training examples) is called a parametric model. No matter how much data you throw at a parametric model, it won’t change its mind about how many parameters it needs.

For nonparametric models ask yourself a question: What is the number of parameters of the decision tree?

As the decision tree is an example of nonparametric model the number of parameters in a decision tree depends on the quantity of the data. The more data we have means more parameters.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .