1
$\begingroup$

The universal approximation theorem states that a feed-forward neural network with a single hidden layer containing a finite number of neurons can approximate a wide variety of interesting (continuous) functions (provided a few assumptions on the activation function are met).

Is there any other machine learning model (apart from any neural network model) that has been proved to be an universal function approximator (and that is potentially comparable to neural networks, in terms of usefulness and applicability)?

$\endgroup$
  • $\begingroup$ I slightly modified my question because, meanwhile, I found these questions on Quora, https://qr.ae/TWI6KL and https://qr.ae/TWI62P, and this question on CS Theory SE cstheory.stackexchange.com/q/7894/34637. I am looking for answers that ideally provide links to references that prove the universality (in terms of function approximation) of the mentioned models. $\endgroup$ – nbro May 10 at 13:38
0
$\begingroup$

In the paper A Note on the Universal Approximation Capability of Support Vector Machines (2002) by Barbara Hammer and Kai Gersmann, the universal function approximation capabilities of SVMs are investigated, where the authors show that SVMs with standard kernels (including Gaussian, polynomial and several dot product kernels) can approximate any measurable or continuous function up to any desired accuracy. Hence, SVMs provide valid generalisation, are universal approximators and are also efficiently trainable.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.