12 votes
Accepted

What are the implications of the "No Free Lunch" theorem for machine learning?

This is a really common reaction after first encountering the No Free Lunch theorems (NFLs). The one for machine learning is especially unintuitive, because it flies in the face of everything that's ...
7 votes
Accepted

Are the shortcomings of neural networks diminishing?

Neural Networks have other short comings as well. It takes much longer and far more resources to train a neural network than something like a random forest. So if you need speed of training or are ...
  • 186
6 votes

Are the shortcomings of neural networks diminishing?

Just to add to what has been said in @MikeWise's brilliant answer, All things equal, deep learning models generally rank supreme when compared to other algorithms as the size of the dataset increases:...
5 votes
Accepted

Are PAC learnability and the No Free Lunch theorem contradictory?

There is no contradiction. First, agnostic PAC learnable doesn't mean that the there is a good hypothesis in the hypothesis class; it just means that there is an algorithm that can probably ...
  • 261
1 vote

If a neural network is a universal function approximator, can it have any prior beliefs?

I think your deduction is mostly correct. Neural networks of depth are universal function approximators. This means that in principal, for any function of the form you describe, there's a NN that ...

Only top scored, non community-wiki answers of a minimum length are eligible