4
$\begingroup$

E. Alpaydin, Introduction to ML, 4-th edition, page 46:

Over time, it has been realized in the neural network community that most neural network learning algorithms have their basis in statistics—for example, the multilayer perceptron is another class of nonparametric estimator.

Why is multilayer perceptron a nonparametric model?

$\endgroup$
2
  • 2
    $\begingroup$ The terms "parametric" and "non-parametric" have been used ambiguously off-late, see discussion - stats.stackexchange.com/a/322051/219259. A comment from that answer says " As the wiki-page on nonparametric statistics says, there is no general consensus on what non-parametric means". $\endgroup$ Commented Aug 1, 2022 at 23:05
  • 1
    $\begingroup$ @GaneshTata Towards the end of 2020, there was a joke in the Cross Validated Meta that our new year’s resolution would be to stop using the term “nonparametric” for exactly that reason. $\endgroup$
    – Dave
    Commented Aug 2, 2022 at 15:25

1 Answer 1

7
$\begingroup$

“Nonparametric” is a controversial term in statistics whose definition lacks a consensus.

One definition is that the parameter count increases as the data size increases. This is how kernel density estimation works: for each new point, we add a new kernel. Perhaps a neural network could be seen in this light by allowing the number of neurons to increase as data are added.

One definition is that we allow the data to describe themselves rather than shoehorning them into a particular form that is governed by parameters. A neural network could be seen this way, as the idea is to give a large amount of flexibility to allow the data to extract their own features that involve curvature and variable interactions; we don’t have to specify these like we do in a linear regression or generalized linear model if the model figures them out.

If the author had referred to a neural network as a particular type of nonlinear regression, then the statement would not be controversial.

Because of the lack of consensus on what “nonparametric” means, towards the end of 2020, there was a movement/joke on Cross Validated (Statistics Stack Exchange) that our new year’s resolution would be to stop using the term “nonparametric” for exactly that reason.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .