I can offer two (at first sight, conflicting) perspectives on this:

Firstly:

*If the letter string 'abc' becomes 'abd' what would "doing the same thing" to 'ijk' look like?*

This is just one example of a problem (so-called 'letter-string analogy problems') that is not easily framed as an optimization problem - there is a range of answers that appear compelling to humans, each for its own structurally-specific reason. Some of the subtleties of these kinds of problems are discussed in detail [here](http://cognitrn.psych.indiana.edu/rgoldsto/courses/concepts/copycat.pdf).

Secondly:

Here's a *very* high-level perspective on AGI in which [optimization plays a key part](http://arxiv.org/abs/cs/0309048).

It's not at all clear how these two very different scales of approach might be reconciled. As someone who does optimization research for a living, I'd be inclined to say that, certainly for all *current, practical* purposes, AGI can't really be treated as an optimization problem, since most interesting activities don't readily lend themselves to description via a cost function.