Questions tagged [universal-approximation-theorems]
For questions related to the (different) universal approximation theorems (UATs), for example, in the context of feed-forward neural networks.
15
questions
0
votes
1answer
76 views
Is it possible to predict $x^2$, $\log(x)$, or variable function of $x$ using RNN?
There were some posts that using RNN can predict the next point of the sine wave function with data history.
However, I wondered if it also works on all the functions of $x$, such as $x^2$, $x^3$, $\...
3
votes
1answer
145 views
Is it really possible to create the “Perfect Cylinder” used in Universal Approximation Theorem for 1-hidden layer Neural Network?
There are proofs for the universal approximation theorem with just 1 hidden layer.
The proof goes like this:
Create a "bump" function using 2 neurons.
Create (infinitely) many of these ...
8
votes
0answers
82 views
What is the number of neurons required to approximate a polynomial of degree n?
I learned about the universal approximation theorem from this guide. It states that a network even with a single hidden layer can approximate any function within some bound, given a sufficient number ...
1
vote
1answer
57 views
Can most of the basic machine learning models be easily represented as simple neural network architectures?
I am currently studying the textbook Neural Networks and Deep Learning by Charu C. Aggarwal. In chapter 1.2.1 Single Computational Layer: The Perceptron, the author says the following:
Different ...
6
votes
1answer
128 views
Smallest possible network to approximate the $sin$ function
The main goal is: Find the smallest possible neural network to approximate the $sin$ function.
Moreover, I want to find a qualitative reason why this network is the smallest possible network.
I have ...
1
vote
1answer
93 views
When are multiple hidden layers necessary?
I know that my question probably seems like being asked many times, but Ill try
to be more speciffic:
Limitations to my question:
I am NOT asking about convolutional neural networks, so please, try ...
2
votes
1answer
141 views
How can neural networks approximate any continuous function but have $\mathcal{VC}$ dimension only proportional to their number of parameters?
Neural networks typically have $\mathcal{VC}$ dimension that is proportional to their number of parameters and inputs. For example, see the papers Vapnik-Chervonenkis dimension of recurrent neural ...
1
vote
1answer
74 views
If a neural network is a universal function approximator, can it have any prior beliefs?
Let us confine ourselves to the case where we have a $n$ dimensional input and a $+1$ or $-1$ output. It can be shown that:
For every $n$, there exists a dense NN of depth 2, such that it contains ...
2
votes
0answers
60 views
Are No Free Lunch theorem and Universal Approximation theorem contradictory in the context of neural networks?
To my understanding NFL states that, we cannot have an hypothesis (let's assume it is an approximator like NN in this case) class that can't achieve certain accuracy parameters $\leq \epsilon$ with ...
4
votes
1answer
111 views
Does a neural network exist that can learn every possible training data?
The universal approximation theorem states, that a feed-forward network with a single hidden layer containing a finite number of neurons can approximate continuous functions on compact subsets of $R^n$...
15
votes
3answers
4k views
Where can I find the proof of the universal approximation theorem?
The Wikipedia article for the universal approximation theorem cites a version of the universal approximation theorem for Lebesgue-measurable functions from this conference paper. However, the paper ...
2
votes
3answers
103 views
Is there a way to calculate the closed-form expression of the function that a neural network computes?
As stated in the universal approximation theorem, a neural network can approximate almost any function.
Is there a way to calculate the closed-form (or analytical) expression of the function that a ...
5
votes
1answer
909 views
Which machine learning models are universal function approximators?
The universal approximation theorem states that a feed-forward neural network with a single hidden layer containing a finite number of neurons can approximate any continuous function (provided some ...
4
votes
3answers
1k views
Is there a mathematical proof that shows that certain parameters work “better” than others for a certain task?
The machine learning community often only provides empirical results, but I am also interested in theoretical results and proofs. Specifically, is there a mathematical proof that shows that certain ...
10
votes
2answers
579 views
What are the learning limitations of neural networks trained with backpropagation?
In 1969, Seymour Papert and Marvin Minsky showed that Perceptrons could not learn the XOR function.
This was solved by the backpropagation network with at least one hidden layer. This type of network ...
asked Aug 3 '16 at 18:05
S.L. Barth - Reinstate Monica
1,1141 gold badge7 silver badges19 bronze badges
