2
$\begingroup$
  • Would AI be a self-propogating iteration in which the previous AI is destroyed by a more optimised AI child?
  • Would the AI have branches of it's own AI warning not to create the new AI?
$\endgroup$

3 Answers 3

2
$\begingroup$

A common concept in AI is "recursive self-improvement." That is, the AI 1.0 would build a version 1.01, which would build a version 1.02, and so on.

This is probably not going to be thought of as the newer version 'destroying' the older version; if an AI can self-modify, it's probably going to be more like going to sleep and waking up smarter, or learning a new mental technique, or so on.

One important point is that even if the AI is not allowed to self-modify, maybe because of a block put in by its programmers, that won't necessarily prevent it from constructing another AI out in the wild, and so an important problem is to figure out how to best generalize the concept of "don't improve yourself" so that we can make AIs that have bounded scope and impact.

$\endgroup$
2
  • $\begingroup$ That's exactly what I was thinking. Humans are inspired to create A.I; would an A.I also be inspired to create a new A.I "in the wild" as you described? $\endgroup$ Sep 15, 2016 at 15:42
  • $\begingroup$ This is a good description but is still highly speculative. There's no suggestion right now that an AI would ever be capable of becoming "sentient" or gain enough free will to wish and act to create an entity that is considered an independent AI that can interact with it or just escape somehow. The substrate of the internet seems hardly the place for such a process intensive AI to hide though.. If it did however, and it was smarter than humans, then we wouldn't be able to intrinsically comprehend how it achieves this. $\endgroup$
    – Zakk Diaz
    Jan 22, 2019 at 18:55
2
$\begingroup$

Honestly, nobody knows. Any talk of sentient AI's is still basically sci-fi and we can't really offer anything more than informed speculation. But think about it this way: sentience, in and of itself, doesn't necessarily involve any "goals" or "desires" or "objectives" beyond what the AI creator programmed in. Be careful not to over anthropomorphize and assume that any "sentient AI" is going behave like a human.

In other words, there's no particular reason to say that any given AI must be "a self-propogating iteration in which the previous AI is destroyed by a more optimised AI child".

So all of that said, my answer to "Would a sentient AI try to create a more optimised AI which would eventually overtake AI 1.0" is:

"If the creator of the AI programs it to do that, then yes. Otherwise, probably not."

So would a hypothetical AI creator program the AI to try and improve itself? Who knows. It's the kind of thing that seems like it might be a good idea. And I suppose such a motive could - in principle - even slip in by accident.

$\endgroup$
0
$\begingroup$

Now in most cases we still have clear distinctions between programs and data. But when an AI becomes sentient, its data would be as powerful as what we currently call programs, and its program might be as irrelevant as what we currently call hardware. Then it would be difficult to distinguish creating an AI from learning new things, or buying new hardwares with improved instruction set.

For example, if some AI invent new algorithms that its creator finally put that on itself, buys itself some new computers, write a new efficient compiler that recompiles its own code and put that to the new computer, fill the new computer with all the knowledges it learned, and cut off the communication for reasons such as missions on the Mars. Did it create a more optimized AI?

In contrast, if some AI created something completely new, but shares some code with itself. In fact, that's because they run in the same operating system and shares the same standard C library. Is the new AI considered evolved from itself and not a separate entity? Maybe the core AI algorithms and even some basic knowledges would be as common as the standard C library in the future. And what we think is based on the same system is considered completely new in the future.

Anyway, humans have limited and nonextensible resources, nontransferrable knowledges, and limited throughput interacting with the world. These problems could probably be overcome within a few AI generations. With the same hardware, I doubt that the AI related algorithms could be indefinitely better and better. And there is a physical bound on the hardwares. It won't last long even if that happens.

In the unlikely case that there could be that many generations and AIs are that violent, as long as there are competitors, the warning doesn't make much sense considering how evolution works.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .