Must people tell an AI which algorithm it should use? Can an AI learn algorithms by itself?

Question

I'm a freshman to machine learning. We all know that there are 2 kinds of problems in our life: problems that humans can solve and problems we can't solve. For problems humans can solve, we always try our best to write some algorithm and tell machine to follow it step by step, and finally the machine acts like people.

What I'm curious about are these problems humans can't solve. If humans ourselves can't sum up and get an algorithm (which means that we ourselves don't know how to solve the problem), can a machine solve the problem? That is, can the machine sum up and get an algorithm by itself based on a large amount of problem data?

Answer 1

All intelligence, both human and machine, is mechanistic. Thoughts don't appear out of the blue; they're generated through specific processes.

This means that if a machine generates an algorithm to solve a problem, even if the object-level algorithm wasn't generated by humans, the meta-level algorithm by which it generated the object-level algorithm must have come from somewhere, and that somewhere is probably its original creators. (Even if they didn't program the meta-level algorithm, they probably programmed the meta-meta-level algorithm that programmed the meta-level algorithm, and so on.)

How you think about these distinctions depends on how you think about machine learning, but typically they're fairly small. For example, when we train a neural network to classify images, we aren't telling it what pixels to focus on or how to combine them, which is the object-level algorithm that it eventually generates. But we are telling it how to construct that object-level algorithm from training data, what I'm calling the 'meta-level' algorithm.

One of the open problems is how to build the meta-meta-level; that is, an algorithm that will be able to look at a dataset and determine which models to train, and then which model to finally use. This will, ideally, include enough understanding of those meta-level models to construct new ones as needed, but even if it doesn't will reflect a major step forward in the usability of ML.

Answer 2

There are problems we for which we don't have a known, optimal, deterministic algorithm. By and large we use heuristics to "solve" those problems. A closely related idea is that of satisficing where we seek out answers that are "good enough" for immediate purposes.

Likewise, machines can also use heuristics, whether they are programmed in explicitly or, presumably, learned. Within the range of ways that a machine can use heuristics, there are meta heuristics and hyper heuristics.

Going beyond that, there are other ways that machines an learn "algorithms" or "rules" for solving problems. One are that I'm particularly interested in is known as rule induction.

This is all an area of open and active research BTW... so if you're interested in exploring any of these approaches, you'll probably find a lot of ground to cover.

Answer 3

I'd like to offer also a slightly different view on the machine cannot better its master. Consider the very simple case of content classifiers. It's already to the point where for some areas classification and prediction can be performed way better than a human. And while a human may have designed the "algorithm", the algorithm was likely a recurrent neutral network or other form of ML that could well have self trained. In these cases we don't actually understand or need to understand the individual weights in the net, as we would need to have traditionally understood the imperative programming constructs we used to write. It just works.

So if we get to where we develop a meta-algorithm for classifying problems and building more optimal deep learning solutions than we would by hand, but I think that would pretty much take us out of the picture for quite a lot of problem spaces. Thoughts?

Answer 4

New guy here, please go easy on me as this answer will come from personal experience, and will probably be a tad philosophical.

Every algorithm I've designed was built to systematically tackle and solve specific problems in specific situations, each with an end goal in mind. Think of algorithms as solutions to a problem. In my career as a programmer, this rule has always stuck with me (it came from my favorite Computer Sciences professor): "If there is no solution, then there is no algorithm. If there is no algorithm, no machine can solve the problem."

Can machines generate their own algorithms? Most likely. But not to the point that it will exceed us (and by exceed, I don't mean just speed). AIs can never solve problems using methods that humans will never be able to come up with, because we programmed AIs to solve problems just like us humans do.