A lot of experts have expressed concerns about evil super intelligence. While their concerns are valid, is it necessary, what are the chances or how the artificial super-intelligence will evolve to have selfishness and self protecting desires inherent in biological systems? Is there any work which comments on this line of inquiry?
AI will only "evolve" selfishness if it "evolves" in a competitive environment and has certain human-like faculties.
Self-protecting desires on the other hand are logical consequences of having any goal at all. After all, you can't reach your goal if you are destroyed.
The concern of "evil" super intelligences isn't that they literally turn evil and selfish and cruel. Those are human qualities.
Instead a superintelligence that has a certain goal, will logically pursue subgoals that help it to reach the ultimate goal. Such subgoals will be resources, power, safety.
So it will amass power and resources to reach its goal and exterminate any threat to its existence as long as its goal hasn't been reached, without being selfish at all.
I have some comments on this subject here: https://ai.stackexchange.com/a/2878/1671
This is some deep game theoretic stuff, and it partly depends on how you define "selfishness".
There is such a thing, for instance, as a "greedy algorithm". Sometimes a greedy algorithm is the most convenient way to achieve an acceptable result, but the optimality is only local.
On a mathematical level, the constructiveness or destructiveness of "self interest" in a system may be a function of whether a Nash Equilibrium is perceived. In this case, self-interest is defined as maintaining the current strategy, because, unless the competitor changes their strategy, there is no gain for changing one's own strategy.
As the BlindKungFuMaster importantly notes, competitiveness will evolve as a trait if the AI operates in a partisan context.
There, the problem comes from whether the "game" is zero sum (Pareto Optimal) or non zero sum (Pareto Improvable), or both. Here, "destructive" may defined agents made worse off per the gains made by another agent.
Altruism seems to have a rational basis, and occurs in evolution because it is presumably useful. [See Biological Altruism.]
Although this tends to be confined to single species, the co-evolution of dogs and humans is a case for inter-species altruism, based on self interest.
Humans and machines have also, and will continue to, co-evolve.