Is increasing software complexity the most likely bottleneck to the AI singularity?

Question

From Wikipedia:

According to the most popular version of the singularity hypothesis, called intelligence explosion, an upgradable intelligent agent will eventually enter a "runaway reaction" of self-improvement cycles, each new and more intelligent generation appearing more and more rapidly, causing an "explosion" in intelligence and resulting in a powerful superintelligence that qualitatively far surpasses all human intelligence.

But what if the complexity of the problem of self-improving the software grows at a faster rate than the AGI intelligence self-improvement?

From experience we know that problems tend to be harder to solve at every iteration, with diminishing returns. Take as an example the theory of gravitation. Newtonian physics is relatively easy to formulate and covers the majority of high level gravitation phenomenas. A more refined picture, like General Relativity, fill few holes in the theory with a huge increase in complexity. To describe black holes and primordial cosmology we need a theory of quantum gravity, which appears to require a further step in complexity.

What "saved" us so far is the economic growth of our civilisation, which allowed more and more scientist to focus on solving the next problems. It's true that the first AGI will have the luxury of being duplicated, being a software base intelligence, but at the same time is likely that the first AGI will be extremely compute intensive. But even assuming that we (or maybe it would better to say, they) have the hardware to run $10^{2}$ instances, if the complexity of every substantial self-improvement grows say by $10^{d}$x with $d=3$ while the improvement to the intelligence is only $10^{l}$x with $l=1$, the self-improvement cycle will quickly slow down.

So is increasing software complexity the most likely bottleneck to the AI singularity? And what are likely values for $d$ and $l$?

Answer 1

What if the complexity of the problem of self-improving the software grows at a faster rate than the AGI intelligence self-improvement?

If this turns out to be the case, then the complexity would indeed be the bottleneck.

There's a formal term for this: kolmogrov complexity or "k-complexity". (The best formal definition of intelligence I've come across is Hutter & Legg's, which is broken down in the article "On the definition of intelligence" by our own nbro. It will shed shed some light on the formal structure of the question at hand.

• Any attempt to provide values to your d & l would speculative, since we're still so far from AGI. The question would be what is the effect of any given (d,l)

The subject can definitely be treated formally in a mathematical sense the and values analyzed. (There may even been some formulae out there, although, they are likely to be more complex, per the Hutter&Legg.)

Sorry I can't give you a more formal answer--it's a good question!

Notes:

Another bottleneck would be the speed of light, essentially, "bits and bytes across a pipeline". This would be a factor in both self-replication and networked processing and memory. You can expand bandwidth/throughput, but not infinitely.

This relates to space complexity (how much volume/memory does the algorithm require?) in addition to the k-complexity, which itself is likely to be non-trivial for anything approaching AGI.

AI Threat:

From the standpoint of neoluddism, I'm skeptical that "infinitely" self-optimizing superintelligence is the greatest threat. Evolution, and economic theory, suggest that the greater danger would come from not what is most expensive, complex, & capable, but that which is "just good enough", cheaper and more easily replicable. This is because decision time becomes a critical factor in a competitive environment, as does replication time.

I always bring up grey goo because it only has to be good at 1 thing to eat everything else!