# What are the differences between artificial neural networks and other function approximators?

Modern artificial neural networks use a lot more functions than just the classic sigmoid, to the point I'm having a hard time really seeing what classifies something as a "neural network" over other function approximators (such as Fourier series, Bernstein polynomials, Chebyshev polynomials or splines).

So, what makes something an artificial neural network? Is there a subset of theorems that apply only to neural networks?

Backpropagation is classic, but that is the multi-variable chain rule, what else is unique to neural networks over other function approximators?

There are other technical differences. For example, the Taylor expansion of a function is typically done only at a single value of the domain, it assumes that the function to be approximated is differentiable multiple times, and it makes uses of the derivatives of such a function. Fourier series typically approximate functions with a weighted sum of sinusoids. Given appropriate weights, the Fourier series can be used to approximate an arbitrary function in a certain interval or the entire function (if the function you want to approximate is also periodic). On the other hand, neural networks attempt to approximate functions of the form $$f: [0, 1]^n \rightarrow \mathbb{R}$$ (at least, this is the setup in the famous paper that proved the universality of neural networks) in many different ways (for example, weighted sums followed by sigmoids).