A big class of problems that are relevant in today's society are full of uncertainty and are also sometimes computationally intractable. Along our lives we come to realize that we are solving the same type of problem multiple times, sometimes with different strategies and mixed results. I would like to close in on three main problem types: pattern recognition, regression and density estimation.
An agent (computer program or even a human) that identifies the type of a problem and applies a systematic procedure for finding its solution. A solution is understood in the classical sense for each of the problem types, thus, the solution does not have to be a global optima. This procedure must be implementable.
Bonus points
- Uses metadata about the problem itself to 'gain insight' about the nature of the problem.
- Verifies that its solution is correct in some sense.
- The types or classes of problems can be expanded later on.
- Works with very limited resources.
- Works with little information about the problem, or with "small data".
So far, I've found Statistical Learning theory and Bayesian Inference as candidates that implement some of those ideas, but I was wondering if there's something else out there or I just need to take the best of both of those worlds.
