Questions tagged [generalization]

For questions related to the concept of generalization in computational learning theory and machine learning.

Filter by
Sorted by
Tagged with
2
votes
0answers
37 views

Why does learning rate reduce train-test generalization gap?

In this blog post: http://www.argmin.net/2016/04/18/bottoming-out/ Prof Recht shows two plots: He says one of the reasons the plot below has a lower train-test gap is because that model was trained ...
2
votes
1answer
39 views

Is there a notion of generalization in unsupervised learning?

I've been learning a little bit about generalization theory, and in particular, the PAC (and PAC-Bayes) approach to thinking about this problem. So, I started to wonder if there is an analogous ...
1
vote
2answers
88 views

Generalization error: Inputs drawn from distributions

I am currently studying Deep Learning by Goodfellow, Bengio, and Courville. In chapter 5.2 Capacity, Overfitting and Underfitting, the authors say the following: Typically, when training a machine ...
1
vote
1answer
43 views

Why does the discrepancy measure involve a supremum over the hypothesis space?

I am referring specifically to the disc defined by Kuznetsov and Mohri in https://arxiv.org/pdf/1803.05814.pdf This is a kind of worst case path dependent generalization error. But what is the ...
2
votes
0answers
39 views

How can we prove this inequality, related to the generalization error, without using the Rademacher complexity?

This is an inequality on page 36 of the book Foundations of Machine Learning, but the author only states it without proof. $$ \mathbb{P}\left[\left|R(h)-\widehat{R}_{S}(h)\right|>\epsilon\right] \...
8
votes
3answers
215 views

Why can neural networks generalize at all?

Neural networks are incredibly good at learning functions. We know by the universal approximation theorem that, theoretically, they can take the form of almost any function - and in practice, they ...
4
votes
1answer
111 views

When exactly is a model considered over-parameterized?

When exactly is a model considered over-parameterized? There are some recent researches in Deep Learning about the role of over-parameterization toward generalization, so it would be nice if I can ...
2
votes
0answers
24 views

References on generalization theory and mathematical abstraction of ML concepts

I'd like to learn about generalization theory for machine learning algorithms. I'm looking for books and other references (in case books aren't available) that provide a gentle introduction to the ...
1
vote
0answers
23 views

Greedy Algorithm with State modelling

My Question Would it be viable to implement and also how would I go about implementing something like this, "AI Agent Rating State-machine"? Problem and constraints: An AI Agent is tasked ...
11
votes
1answer
257 views

What are the state-of-the-art results on the generalization ability of deep learning methods?

I've read a few classic papers on different architectures of deep CNNs used to solve varied image-related problems. I'm aware there's some paradox in how deep networks generalize well despite ...
1
vote
0answers
66 views

How do I determine the generalisation ability of a neural network?

I am trying to ascertain if my neural network is able to generalize or if it’s simply using memory/overfitting to solve a task. I would like my model to generalise. Currently, I train the neural ...
3
votes
2answers
810 views

How can generalization error be estimated?

How would you estimate the generalisation error? What are the methods of achieving this?
11
votes
3answers
659 views

How does noise affect generalization?

Does increasing the noise in data help to improve the learning ability of a network? Does it make any difference or does it depend on the problem being solved? How is it affect the generalization ...