Questions tagged [generalization]

For questions related to the concept of generalization in computational learning theory and machine learning.

Filter by
Sorted by
Tagged with
2
votes
0answers
36 views

Why does learning rate reduce train-test generalization gap?

In this blog post: http://www.argmin.net/2016/04/18/bottoming-out/ Prof Recht shows two plots: He says one of the reasons the plot below has a lower train-test gap is because that model was trained ...
2
votes
1answer
38 views

Is there a notion of generalization in unsupervised learning?

I've been learning a little bit about generalization theory, and in particular, the PAC (and PAC-Bayes) approach to thinking about this problem. So, I started to wonder if there is an analogous ...
1
vote
2answers
85 views

Generalization error: Inputs drawn from distributions

I am currently studying Deep Learning by Goodfellow, Bengio, and Courville. In chapter 5.2 Capacity, Overfitting and Underfitting, the authors say the following: Typically, when training a machine ...
1
vote
1answer
42 views

Why does the discrepancy measure involve a supremum over the hypothesis space?

I am referring specifically to the disc defined by Kuznetsov and Mohri in https://arxiv.org/pdf/1803.05814.pdf This is a kind of worst case path dependent generalization error. But what is the ...
2
votes
0answers
38 views

How can we prove this inequality, related to the generalization error, without using the Rademacher complexity?

This is an inequality on page 36 of the book Foundations of Machine Learning, but the author only states it without proof. $$ \mathbb{P}\left[\left|R(h)-\widehat{R}_{S}(h)\right|>\epsilon\right] \...
8
votes
3answers
210 views

Why can neural networks generalize at all?

Neural networks are incredibly good at learning functions. We know by the universal approximation theorem that, theoretically, they can take the form of almost any function - and in practice, they ...
4
votes
1answer
98 views

When exactly is a model considered over-parameterized?

When exactly is a model considered over-parameterized? There are some recent researches in Deep Learning about the role of over-parameterization toward generalization, so it would be nice if I can ...
2
votes
0answers
23 views

References on generalization theory and mathematical abstraction of ML concepts

I'd like to learn about generalization theory for machine learning algorithms. I'm looking for books and other references (in case books aren't available) that provide a gentle introduction to the ...
1
vote
0answers
23 views

Greedy Algorithm with State modelling

My Question Would it be viable to implement and also how would I go about implementing something like this, "AI Agent Rating State-machine"? Problem and constraints: An AI Agent is tasked ...
11
votes
1answer
245 views

What are the state-of-the-art results on the generalization ability of deep learning methods?

I've read a few classic papers on different architectures of deep CNNs used to solve varied image-related problems. I'm aware there's some paradox in how deep networks generalize well despite ...
1
vote
0answers
66 views

How do I determine the generalisation ability of a neural network?

I am trying to ascertain if my neural network is able to generalize or if it’s simply using memory/overfitting to solve a task. I would like my model to generalise. Currently, I train the neural ...
3
votes
1answer
547 views

How can generalization error be estimated?

How would you estimate the generalisation error? What are the methods of achieving this?
11
votes
3answers
637 views

How does noise affect generalization?

Does increasing the noise in data help to improve the learning ability of a network? Does it make any difference or does it depend on the problem being solved? How is it affect the generalization ...