Skip to main content
14 votes
Accepted

What are the implications of the "No Free Lunch" theorem for machine learning?

This is a really common reaction after first encountering the No Free Lunch theorems (NFLs). The one for machine learning is especially unintuitive, because it flies in the face of everything that's ...
John Doucette's user avatar
6 votes

Are the shortcomings of neural networks diminishing?

Just to add to what has been said in @MikeWise's brilliant answer, All things equal, deep learning models generally rank supreme when compared to other algorithms as the size of the dataset increases:...
Tshilidzi Mudau's user avatar
6 votes
Accepted

Are the shortcomings of neural networks diminishing?

Neural Networks have other short comings as well. It takes much longer and far more resources to train a neural network than something like a random forest. So if you need speed of training or are ...
Mike Wise's user avatar
  • 176
6 votes
Accepted

Are PAC learnability and the No Free Lunch theorem contradictory?

There is no contradiction. First, agnostic PAC learnable doesn't mean that the there is a good hypothesis in the hypothesis class; it just means that there is an algorithm that can probably ...
Anon's user avatar
  • 271
2 votes
Accepted

No-Free-Lunch: Calculation of the number of sequences of examples of size $m$

Your expression is "how many ways can I choose m unique elements from a list of 2m unique elements" The author's expression is "how many unique sequences of length m can I construct ...
Neil Slater's user avatar
  • 32.7k
1 vote

If a neural network is a universal function approximator, can it have any prior beliefs?

I think your deduction is mostly correct. Neural networks of depth are universal function approximators. This means that in principal, for any function of the form you describe, there's a NN that ...
John Doucette's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible