In this video an expert says, "One way of thinking about what intelligence is [specifically with regard to artificial intelligence], is as an optimization process."

Can intelligence always be thought of as an optimization process, and can artificial intelligence always be modeled as an optimization problem? What about pattern recognition? Or is he mischaracterizing?


2 Answers 2


A good answer to this question depends on what you want to use the labels for.

When I think about "optimization," I think about a solution space and a cost function; that is, there are many possible answers that could be returned and we can know what the cost is of any particular answer.

In this view, the answer is "yes"--pattern recognition is a case where each pattern is a possible answer, and the optimization method is trying to find the one where the cost is lowest (that is, where the answer matches what you want it to match).

But most interesting optimization problems are characterized by exponential solution spaces and clean cost functions, and so can be thought of more as 'search' problems, whereas most pattern recognition problems are characterized by simple solution spaces and complicated cost functions, and it might feel unnatural to put the two of them together.

(In general, I do think that optimization and intelligence are deeply linked enough that optimization power is a good measure of intelligence, and certainly a better measure of the practical use of intelligence than pattern recognition.)


I can offer two (at first sight, conflicting) perspectives on this:


If the letter string 'abc' becomes 'abd' what would "doing the same thing" to 'ijk' look like?

This is just one example of a problem (so-called 'letter-string analogy problems') that is not easily framed as an optimization problem - there is a range of answers that appear compelling to humans, each for its own structurally-specific reason. Some of the subtleties of these kinds of problems are discussed in detail here.


Here's a very high-level perspective on AGI in which optimization plays a key part.

It's not at all clear how these two very different scales of approach might be reconciled. As someone who does optimization research for a living, I'd be inclined to say that, certainly for all current, practical purposes, AGI can't really be treated as an optimization problem, since most interesting activities don't readily lend themselves to description via a cost function.

  • $\begingroup$ Your first example is interesting. I haven't yet read that linked article, but are you sure that this problem cannot be successfully solved by framing it as an optimization problem (i.e. can't we solve the "analogy" problem as an optimization problem)? It would be nice if you provided more details about this part "there is a range of answers that appear compelling to humans, each for its own structurally-specific reason". I guess the details can be found in the linked article, but you could make this interesting answer more self-contained. $\endgroup$
    – nbro
    Commented Jan 25, 2021 at 16:00

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .