Linked Questions

31 votes
3 answers
16k views

Where can I find the proof of the universal approximation theorem?

The Wikipedia article for the universal approximation theorem cites a version of the universal approximation theorem for Lebesgue-measurable functions from this conference paper. However, the paper ...
Leroy Od's user avatar
  • 475
5 votes
4 answers
2k views

How does size of the dataset depend on VC dimension of the hypothesis class?

This might be a little broad question, but I have been watching Caltech youtube videos on Machine Learning, and in this video prof. is trying to explain how we should interpret the VC dimension in ...
Stefan Radonjic's user avatar
12 votes
3 answers
503 views

Are there any rules of thumb for having some idea of what capacity a neural network needs to have for a given problem?

To give an example. Let's just consider the MNIST dataset of handwritten digits. Here are some things which might have an impact on the optimum model capacity: There are 10 output classes The inputs ...
Alexander Soare's user avatar
3 votes
3 answers
1k views

How would you intuitively but rigorously explain what the VC dimension is?

The VC dimension is a very important concept in computational/statistical learning theory. However, the first time you read its definition, you may not immediately understand what it really represents ...
nbro's user avatar
  • 40.8k
5 votes
1 answer
773 views

How can I estimate how many photos I need to train ResNet-50 for image classification?

I am working on a project where I have to classify around 1000 unique objects. I'm trying to plan how much training data I will need to collect. I was planning on using ResNet-50. Is there anyway I ...
Tyler Hilbert's user avatar
4 votes
2 answers
420 views

Mathematical foundations of the ability to learn

I am an undergraduate student in applied mathematics with an interest in artificial intelligence. I am currently exploring topics where I could do research. Coming from a mathematical background I am ...
Matheo's user avatar
  • 143
1 vote
1 answer
1k views

What are the conceptual differences between regularisation and optimisation in deep neural nets?

I'm trying to wrap my mind around the concepts of regularisation and optimisation in neural nets, especially around their differences. In my current understanding, regularisation is intended to tackle ...
Felipe Martins Melo's user avatar
2 votes
2 answers
987 views

Why don't neural networks project the data into higher dimensions first, then reduce the size of each layer thereafter?

Background From my understanding (and following along with this blog post), (deep) neural networks apply transformations to the data such that the data's representation to the next layer (or ...
Kevin's user avatar
  • 133
1 vote
1 answer
449 views

What do we mean by saying "VC dimension gives a LOOSE, not TIGHT bound"?

From what I understand VC dimension is what establishes the feasibility of learning for infinite hypothesis sets, the only kind we would use in practice. But, the literature (i.e. Learning from Data)...
Stefan Radonjic's user avatar
2 votes
1 answer
431 views

How does the number of stacked LSTM layers or units in each layer affect the model complexity?

I playing around sequence modeling to forecast the weather using LSTM. How does the number of layers or units in each layer exactly affect the model complexity (in an LSTM)? For example, if I ...
Manojk07's user avatar
  • 121
2 votes
2 answers
82 views

Is there a measure of model complexity?

Is there a measure of model complexity?
Justaperson's user avatar