3
$\begingroup$

It seems to me that the first AGIs ought to be able to perform the same sort and variety of tasks as people, with the most computationally strenuous tasks taking an amount of time compared to how long a person would take. If this is the case, and people have yet to develop basic AGI (meaning it's a difficult task), should we be concerned if AGI is developed? It would seem to me that any fears about a newly developed AGI, in this case, should be the same as fears about a newborn child.

Improve this question
$\endgroup$
1
  • $\begingroup$ We know how children work better than we know how AGI works. $\endgroup$
    – user253751
    Feb 15 at 18:50

3 Answers 3

Reset to default
4
$\begingroup$

There are basically two worries:

If we create an AGI that is a slightly better AGI-programmer than its creators, it might be able to improve its own source code to become even more intelligent. Which would enable it to improve its source code even more etc. Such a selfimproving seed AI might very quickly become superintelligent.

The other scenario is that intelligence is such a complicated algorithmic task, that when we finally crack it, there will be a significant hardware overhang. So the "intelligence algorithm" would be human level on 2030 hardware, but we figure it out in 2050. In that case we would immediately have superintelligent AI without ever creating human level AI. This scenario is especially likely because development often requires a lot of test runs to tweak parameters and try out different ideas.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ In reference to the first option: why can't we make now a piece of relatively simple software that is able to improve its own code, so it would spiral out from current constraint of waiting for people to create machine slightly better than themselves? $\endgroup$
    – Alex
    Feb 23, 2018 at 15:05
  • $\begingroup$ Please direct me to a full question regarding this if there is one. Are there ideas out there (say, in Bostrom's "Superintelligence") that detail how to kill the AGI before it kills us, supposing we can keep it contained to, for instance, one machine that we've kept detached from the internet? $\endgroup$
    – Nathan majicvr.com
    Jul 7, 2019 at 4:23
1
$\begingroup$

To avoid a repetitive answer that has been already spoken about such as absurdly high iterative ability or it being able to create another AGI system and multiplying or anything sci-fi like - there is one line of thought I feel people do not speak enough about.

Our human senses are extremely limited i.e. we can see objects only when light from within the visible light spectrum (~ 400nm-700nm) reflects into our eyes, we can hear only a limited range of frequencies the rest being inaudible etc. An AGI system apart from its obvious intelligence, would be able to gain a significant amount of information from even common observations. It can see infrared, ultraviolet and radio waves as what we interpret as colours; it would be able to hear sounds that we did not know were being emitted at all. Essentially an AGI with good input sensor capabilities would be able to take information from experiencing the world as it actually is, and not a limited illusion we experience.

Improve this answer
$\endgroup$
0
$\begingroup$

This depends on the definition(s) of AGI and ASI. Both are currently ill-defined. Most researchers in AGI follow their own definition of AGI.

At least one researcher believes that there is no such thing as ASI. This is because the basic principles of said AGI always stay the same. It may be learning processes, the core logic(s) and the control logic (reasoning systems are divided into control systems and logic systems, the control system(s) decide which derivations are fruitful).

ASI may be defined as a search for any combination of these (just a subset which come to my mind):

  • search for better algorithms
  • heuristics
  • better contemporary (NN) architectures
  • learning mechanisms
  • solving techniques
  • higher subjective beauty
  • better compression of knowledge
  • better subsystems
  • NN and in general architectures
  • better embedded AGI's
  • faster solving capabilities of known problems
  • ...

There are limitations to any sort of (recursive) self improvement however. Examples of these are

  • the score of AlphaGo and AlphaGo-Zero plateaus after a long enough training period
  • supercompilation of a supercompiled program yields no improved program after a few iterations
  • ...

Note here that these are examples about weak-AI and may not apply to AGI - but it is very likely in my opinion.

So the level of worry depends on the plausible (or followed) definition of AGI and the assumptions of the mechanisms an AGI may employ.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .