Questions tagged [universal-approximation-theorems]
For questions related to the (different) universal approximation theorems (UATs), for example, in the context of feed-forward neural networks.
7
questions
6
votes
1answer
62 views
+100
Smallest possible network to approximate the $sin$ function
The main goal is: Find the smallest possible neural network to approximate the $sin$ function.
Moreover, I want to find a qualitative reason why this network is the smallest possible network.
I have ...
1
vote
1answer
76 views
When are multiple hidden layers necessary?
I know that my question probably seems like being asked many times, but Ill try
to be more speciffic:
Limitations to my question:
I am NOT asking about convolutional neural networks, so please, try ...
2
votes
1answer
103 views
How can neural networks approximate any continuous function but have $\mathcal{VC}$ dimension only proportional to their number of parameters?
Neural networks typically have $\mathcal{VC}$ dimension that is proportional to their number of parameters and inputs. For example, see the papers Vapnik-Chervonenkis dimension of recurrent neural ...
2
votes
0answers
48 views
Are No Free Lunch theorem and Universal Approximation theorem contradictory in the context of neural networks?
To my understanding NFL states that, we cannot have an hypothesis (let's assume it is an approximator like NN in this case) class that can't achieve certain accuracy parameters $\leq \epsilon$ with ...
11
votes
2answers
3k views
Where can I find the proof of the universal approximation theorem?
The Wikipedia article for the universal approximation theorem cites a version of the universal approximation theorem for Lebesgue-measurable functions from this conference paper. However, the paper ...
4
votes
1answer
663 views
Which machine learning models are universal function approximators?
The universal approximation theorem states that a feed-forward neural network with a single hidden layer containing a finite number of neurons can approximate any continuous function (provided some ...
3
votes
3answers
1k views
Is there a mathematical proof that shows that certain parameters work “better” than others for a certain task?
The machine learning community often only provides empirical results, but I am also interested in theoretical results and proofs. Specifically, is there a mathematical proof that shows that certain ...
