11

This is a really common reaction after first encountering the No Free Lunch theorems (NFLs). The one for machine learning is especially unintuitive, because it flies in the face of everything that's discussed in the ML community. That said, the theorem is true, but what it means is open to some debate. To restate the theorem for people who don't know it, ...


7

Neural Networks have other short comings as well. It takes much longer and far more resources to train a neural network than something like a random forest. So if you need speed of training or are resource constrained in anyway, you probably should not look at Neural Networks first. Evaluation of a trained deep NN can be much more expensive than competing ...


5

Just to add to what has been said in @MikeWise's brilliant answer, All things equal, deep learning models generally rank supreme when compared to other algorithms as the size of the dataset increases: Like everything, it all boils down to the dataset at hand, neural networks are good on other datasets but at the same time, they will be bad on other ...


4

There is no contradiction. First, agnostic PAC learnable doesn't mean that the there is a good hypothesis in the hypothesis class; it just means that there is an algorithm that can probably approximately do as well as the best hypothesis in the hypothesis class. Also, these NFL theorems have specific mathematical statements, and hypothesis classes for ...


1

I think your deduction is mostly correct. Neural networks of depth are universal function approximators. This means that in principal, for any function of the form you describe, there's a NN that approximates it. However, a particular NN architecture of fixed width and depth, with fixed connections is not a universal approximator for all functions. Only ...


Only top voted, non community-wiki answers of a minimum length are eligible