A lot of experts have expressed concerns about evil super intelligence. While their concerns are valid, is it necessary, what are the chances or how the artificial super-intelligence will evolve to have selfishness and self protecting desires inherent in biological systems? Is there any work which comments on this line of inquiry?
3
-
1$\begingroup$ Possible duplicate of Could an AI feel emotion? $\endgroup$– Mithical ♦Commented Mar 1, 2017 at 10:57
-
$\begingroup$ @Mithrandir So far two question have been duplicated from one question;and this Can Robots learn how to feel suffering? $\endgroup$– quintumniaCommented Mar 1, 2017 at 11:41
-
$\begingroup$ @quintumnia Not necessarily a duplicate imho, because this question is about "self interest" in a game theory context specifically, as opposed to useful traits related to "self interest" that could arise out of game theoretic applications of automata (genetic algorithms and evolutionary game theory.) $\endgroup$– DukeZhouCommented Mar 1, 2017 at 16:35
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
2
AI will only "evolve" selfishness if it "evolves" in a competitive environment and has certain human-like faculties.
Self-protecting desires on the other hand are logical consequences of having any goal at all. After all, you can't reach your goal if you are destroyed.
The concern of "evil" super intelligences isn't that they literally turn evil and selfish and cruel. Those are human qualities.
Instead a superintelligence that has a certain goal, will logically pursue subgoals that help it to reach the ultimate goal. Such subgoals will be resources, power, safety.
So it will amass power and resources to reach its goal and exterminate any threat to its existence as long as its goal hasn't been reached, without being selfish at all.
answered Mar 1, 2017 at 11:42
-
$\begingroup$ Hello BlindKungFuMaster,Can you please first have a look at duplicated questions. $\endgroup$ Commented Mar 1, 2017 at 11:44
-
1$\begingroup$ @quintumnia: I don't think this question is a duplicate of the emotion or of the suffering question. Selfishness is neither an emotion nor a feeling. Also, the other questions are missing the AI-threat aspect. I really don't see much similarity. $\endgroup$ Commented Mar 1, 2017 at 14:35
$\begingroup$
$\endgroup$
I have some comments on this subject here: https://ai.stackexchange.com/a/2878/1671
This is some deep game theoretic stuff, and it partly depends on how you define "selfishness".
There is such a thing, for instance, as a "greedy algorithm". Sometimes a greedy algorithm is the most convenient way to achieve an acceptable result, but the optimality is only local.
On a mathematical level, the constructiveness or destructiveness of "self interest" in a system may be a function of whether a Nash Equilibrium is perceived. In this case, self-interest is defined as maintaining the current strategy, because, unless the competitor changes their strategy, there is no gain for changing one's own strategy.
As the BlindKungFuMaster importantly notes, competitiveness will evolve as a trait if the AI operates in a partisan context.
There, the problem comes from whether the "game" is zero sum (Pareto Optimal) or non zero sum (Pareto Improvable), or both. Here, "destructive" may defined agents made worse off per the gains made by another agent.
Altruism seems to have a rational basis, and occurs in evolution because it is presumably useful. [See Biological Altruism.]
Although this tends to be confined to single species, the co-evolution of dogs and humans is a case for inter-species altruism, based on self interest.
Humans and machines have also, and will continue to, co-evolve.
