Questions tagged [capacity]

For questions related to the often not well-defined or vaguely defined notion of capacity of a neural network (or any other machine learning model). The capacity of a neural network can be precisely defined as its VC dimension.

Filter by
Sorted by
Tagged with
4
votes
2answers
399 views

How to estimate the capacity of a neural network?

Is it possible to estimate the capacity of a neural network model? If so, what are the techniques involved?
9
votes
3answers
367 views

Are there any rules of thumb for having some idea of what capacity a neural network needs to have for a given problem?

To give an example. Let's just consider the MNIST dataset of handwritten digits. Here are some things which might have an impact on the optimum model capacity: There are 10 output classes The inputs ...
7
votes
3answers
1k views

What is the difference between hypothesis space and representational capacity?

I am reading Goodfellow et al Deeplearning Book. I found it difficult to understand the difference between the definition of the hypothesis space and representation capacity of a model. In Chapter 5,...