Is there a trade-off between flexibility and efficiency?

Question

A "general intelligence" may be capable of learning a lot of different things, but possessing capability does not equal actually having it. The "AGI" must learn...and that learning process can take time. If you want an AGI to drive a car or play Go, you have to find some way of "teaching" it. Keep in mind that we have never built AGIs, so we don't know how long the training process can be, but it would be safe to assume pessimistic estimates.

Contrast that to a "narrow intelligence". The narrow AI already knows how to drive a car or play Go. It has been programmed to be very excellent at one specific task. You don't need to worry about training the machine, because it has already been pre-trained.

A "general intelligence" seems to be more flexible than a "narrow intelligence". You could buy an AGI and have it drive a car and play Go. And if you are willing to do more training, you can even teach it a new trick: how to bake a cake. I don't have to worry about unexpected tasks coming up, since the AGI will eventually figure out how to do it, given enough training time. I would have to wait a long time though.

A "narrow intelligence" appears to be more efficient at its assigned task, due to it being programmed specifically for that task. It knows exactly what to do, and doesn't have to waste time "learning" (unlike our AGI buddy here). Instead of buying one AGI to handle a bunch of different tasks poorly, I would rather buy a bunch of specialized narrow AIs. Narrow AI #1 drives cars, Narrow AI #2 plays Go, Narrow AI #3 bake cakes, etc. That being said, this is a very brittle approach, since if some unexpected task comes up, none of my narrow AIs would be able to handle it. I'm willing to accept that risk though.

Is my "thinking" correct? Is there a trade-off between flexibility (AGI) and efficiency (narrow AI), like what I have just described above? Or is it theoretically possible for an AGI to be both flexible and efficient?

Answer 1

The cleanest result we have on this issue is the "no free lunch" theorem. Basically, in order to make a system perform better at a specific task, you have to degrade its performance on other tasks, and so there is a flexibility-efficiency tradeoff.

But to the broader question, or whether or not your thinking is correct, I think it pays to look more closely at what you mean by a "narrow intelligence." The AI systems that we have that play Go and drive cars did not pop into existence able to do those things; they slowly learned how through lots and lots of training examples and a well-chosen architecture that mirrors the problem domain.

That is, "neural networks" as a methodology seems 'general' in a meaningful way; one could imagine that a general intelligence could be formed by solving the meta-learning problem (that is, learning the architecture that best suits a particular problem while learning the weights for that problem from training data).

Even in that case, there will still be a flexibility-efficiency tradeoff; the general intelligence that's allowed to vary its architecture will be able to solve many different problems, but will take some time to discover what problem it's facing. An intelligence locked into a particular architecture will perform well on problems that architecture is well-suited for (better than the general, since it doesn't need to discover) but less well on other problems it isn't as well-suited for.

Answer 2

As Matthew Graves explained in another answer No free lunch theorem confirms the flexibility - efficiency trade-off. However, this theorem is describing a situation where you have a set of completely independent tasks. This often doesn't hold, as many different problems are equivalent in their core or at least have some overlap. Then you can do something called "transfer learning", which means that by training to solve one task you also learn something about solving another one (or possibly multiple different tasks).

For example in Policy Distillation by Rusu et al. they managed to "distill" knowledge from different expert networks into one general network which in the end outperformed each of the experts. The experts were trained for specific tasks while the generalist learned the final policy from these "teachers".

Answer 3

It would appear so. One example, albeit not specifically AI related, is seen in the difference between digital computers and analog computers. Pretty much everything we think of as a "computer" today is a digital computer with a von Neumann architecture. And that's because the things are so general purpose that they can be easily programmed to do, essentially, anything. But analog computers can (or could, back in the 60's or thereabouts) solve some types of problems faster than a digital computer. But they fell out of favor exactly due to that lack of flexibility. Nobody wants to hand-wire circuits with op-amps and comparators to solve for y.