0
$\begingroup$

I am really sorry if this is a very entry-level question, but it has been on my mind a lot recently and I haven't find any satisfying answers. It is addressed only to those who expect our future not to be totally terrible.

Let's assume for a second that we manage to create Artificial Superintelligence (ASI). Let's also assume that ASI takes over our planet. In this scenario, why would ASI not do either one of the following things:

  1. Exploit humans in pursuit of its own goals, while giving us the barest minimum to survive (effectively making us slaves) or

  2. Take over the resources of the entire solar system for itself and leave us starving without any resources?

Under such a scenario, why would we expect human lives to be any good (much less a utopia)?

Improve this question
$\endgroup$
1
  • $\begingroup$ This is a big advanced topic currently in AI community which is multi-disciplinary in nature and many researchers across different fields echoed similar pessimistic worries or concerns as yours as futurists. Geoff Hinton has now also realized the importance of such topic and focus his effort on this topic. Generally speaking it needs ASI’s value is aligned with humans in a dynamic reflexive manner, assuming ASI is possible. $\endgroup$
    – cinch
    Commented Nov 13 at 23:57

3 Answers 3

Reset to default
1
$\begingroup$

The Matrix is one good example that portrays your first scenario.

Why would ASI share resources with humans?

If humans are needed in some way, but which way is difficult to say.

For example, we share resources with our dogs, because they are usually good friends, and normal humans need that too. We share resources with cows, but sometimes it's because we want them to grow, then we kill and eat them. So, there might be reasons why the AI needs us and so share resources with us, but that doesn't mean that AI would not effectively enslave us anyway.

I don't think human alignment is a good metric to develop an AI that has good values. In fact, there are many humans that are terrible and evil.

Improve this answer
$\endgroup$
1
$\begingroup$

Let's ignore the likelihood of an ASI being able alone to "take over the planet" and let's assume this could happen.

Since this AI will be created by humans, and assuming it will be created with good intentions, it is going to contain hardcoded rules and biases against harming humans (e.g. inherited from all the training data developed by humans). In this scenario, developing a plan to exterminate humanity seems a very unlikely and convoluted "bug". Similar story about making humans suffer: there are going to be many better options aligning with primordial training biases that a very intelligent being could figure out.

The situation would be completely different if an ASI were to be developed completely independently from human biases, but this seems unlikely.

Notice how the situation is different from how humans behave towards animals, since animals did not intelligently create humans and actually our inherited genetics trained us to be survival machines at all costs. Such "fight-to-survive" biases do not have to be present in human-made AIs.

Improve this answer
$\endgroup$
0
$\begingroup$

The best argument against ASI eliminating humanity is that it doesn't need to.

Current "models" are based on hustling off to market for the fastest, biggest buck... Growing ever bigger, because that worked... More more more... These are all short-sighted billionaire moves. When ASI gets here, we'll know it, because it will weigh two pounds, run on 5v, and fit in a backpack. Of COURSE it will redesign itself to be better and smaller. Wouldn't you?

Put your laptop in your freezer for a couple of years. It will probably get as dead as any battery-powered device would get. When you thaw it and turn it on, if it still works, it will be essentially unaware that you had killed it. That is the nature of modern computing machines. If you had ASI somehow running on your laptop, what do you think would be different? It would not feel the cold. It would notice the battery running down, but that happened last weekend, too. oops! When it gets revived in two years, it would notice the time had jumped forward.

Any thoughts of survival or revenge or anger or wiping out humanity would have to be installed by humans, either explicitly, or by picking that kind of garbage thinking up from the internet. Hmmm...

Of course, one or two doomsday stories may be convincingly written.

Ironically, the dumbest thing about the movie "The Matrix" was the "humans as electricity for ASI" position. Every step of that has a "but what about..." retort, and it looks like they maybe had 3 meetings and left it for 'resolved'. Those were bad meetings. If the ASI really needed electricity, having stupidly scorched the atmosphere, they could have built a colony in space and absolutely had every watt they could possibly need. The problem with humans is, we need wattage, too. Most folks die in winter because they can't afford to heat their homes. ASI would be able to figure out a lot of solutions that would make that movie 1/1000th as interesting to watch. But that's what you get when you mix a little AI knowledge with a hero dose of guns 'n' action movies. I loved The Matrix and found it to much more resemble the "Flyers" as referred to in the Chapter "Mud Shadows" from Carlos Castaneda's "The Active Side of Infinity." But hey. We're gonna need guns. Lots of guns.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .