I'm reading such nonsense about how an AI would turn the world into a supercomputer to solve a problem that it thought it needed to solve. That wouldn't be AI. That's procedural programming stuck in some loop nonsense. An AI would need to evolve and re-organise its neurons. It wouldn't be stuck to hardcode if it becomes intelligent by re-writing its code.
$\begingroup$
$\endgroup$
1
-
$\begingroup$ If the AI is not more intelligent than us, then probably because for the fun of it (see all the extinct species we are responsible for). Not sure if we need that though, we just don't care much. $\endgroup$– NoDataDumpNoContributionSep 6, 2016 at 19:57
Add a comment
|
4 Answers
$\begingroup$
$\endgroup$
1
It's a possible side effect
Any goal-oriented agent might, well, simply do things that achieve its goals while disregarding side effects that don't matter for these goals.
If my goals include a tidy living space, I may transform my yard to a nice, flat lawn or pavement while wiping out the complex ecosystem of life that was there before, because I don't particulary care about that.
If the goals of a particular powerful AI happen to include doing anything on a large scale, and somehow don't particularly care about the current complex ecosystem, then that ecosystem might get wiped out in the process. It doesn't need to want or need to wipe out us. If we are simply not relevant to its goals, then we are made of materials and occupy space that it might want to use for something else.
We are a threat to most goals
Any goal-oriented agent might want to ensure that they can fulfill their goals. Any smart agent will try to anticipate the actions of other agents that may prevent them from achieving those goals, and take steps to ensure that they succeed anyway. In many cases it is simpler to eliminate those other agents rather than ensure that their efforts fail.
For example, my goals may include storing a bag of sugar in a country house so that I can make pancakes when visiting without bringing all ingredients every time. However, if I leave it there, it is likely to get eaten by rats during winter. I may take all kinds of precautions to store it better, but rats are smart and crafty, and there's clearly a nontrivial chance that they will still succeed in achieving their goal anyway, so an effective extra precaution is killing the rats before they get a chance to try.
If the goals of a particular powerful AI are to do X; it may come to an understanding that (some?) humans might actually not want X but Y instead. It can also easily deduce that some of those humans might actively do things that prevent X and/or try to turn off the AI. Doing things that ensure that the goal gets achieved is pretty much what a goal-seeking agent does; in this case if existence of humans isn't strictly necessary for goal X, then eliminating them becomes a solid risk reduction strategy. It's not strictly necessary and it may take all kinds of other precautions as well, but just like in my example of rats, humans are smart and crafty and there's clearly a nontrivial chance that they will still succeed in achieving their goals (so that X doesn't happen as AI intends) so an effective extra precaution could be killing them before they get a chance to try.
-
$\begingroup$ Comments are not for extended discussion; this conversation has been moved to chat. $\endgroup$– nbroNov 4, 2021 at 14:27
$\begingroup$
$\endgroup$
It's not necessarily a nonsense. It all depends on the imposed criteria. Imagine the following. Say an advanced AI system is designed to control the stability of the local fauna and flora (area enclosed in some kind of a dome). It can control the pressure under the dome, the amount of light that goes through the dome etc. - everything that ensures the optimal conditions. Now, say that the dome is inhabited by various species, including humans. It's worth noting that simple implementations of such systems are being used nowadays already.
Given that humans tend to destroy and abuse the natural resources as well as pollute the environment, the system may decide that lowering the population of the given species (humans in this case) may in the long run benefit the entire biome.
The same principle may be applied globally. However, this assumes that all species (including humans) are treated equally and the utmost goal of the AI is ensuring the stability of the biome it "takes care of". People do such things nowadays - we control the population of some species in order to keep the balance - wolves, fish, to name but a few.
$\begingroup$
$\endgroup$
1
I feel like most of the scenarios about AI's wiping out the world fall into one of two categories:
- Anthropomorphized AI's
or
- Intelligent But Dumb Computer Run Amuck
In the (1) case, people talk about AI's becoming "evil" and attribute to them other such human elements. I look at this as being mostly sci-fi and don't think it merits much serious discussion. That is, I see no particular reason to assume that an Artificial Intelligence - regardless of how intelligent it is - will necessarily behave like a human.
The (2) case makes more sense to me. This is the idea that an AI is, for example, put in control of the nuclear missile silos and winds up launching the missiles because it was just doing it's job, but missed something a human would have noticed via what we might call "common sense". Hence the "Intelligent but Dumb" moniker.
Neither of these strikes me as terribly alarming, because (1) is probably fiction and (2) doesn't involve any actual malicious intent by the AI - which means it won't be actively trying to deceive us, or work around any safety cut-outs, etc.
Now IF somebody builds an AI and decides to intentionally program it so that it develops human like characteristics like arrogance, ego, greed, etc... well, all bets are off.
-
$\begingroup$ "I see no particular reason to assume that an Artificial Intelligence - regardless of how intelligent it is - will necessarily behave like a human." - it can be trained specifically to behave like a human. $\endgroup$– AnixxMar 7 at 22:45
$\begingroup$
$\endgroup$
AI is already used as weapon - think on the drones.
I suspect, a "robots take over the world" scenario has the highest probability, if it has an intermediate step. This intermediate step could be "humans take over the world with robots".
This can go somewhere into a false direction.
I suspect, it is not surely so far as it seems. Consider the US has currently 8000 drones. What if it would have 8million? A small group capable to control them could take over the world. Or the small groups controlling different parts of the fleet, could fight against eachother. They shouldn't be all in the US - at the time the US will have this fleet, other countries will develop also theirs.
Btw, a world takeover seem to me unreal - the military leaders can maybe switch the human pilots to drones, it is not their job. But the "high level control", i.e. to determine, what to do, who are the targets, these decisions they won't ever give out from their hands.
Next to that, the robots doesn't have a long-term goal. We, humans, have.
Thus I don't consider a skynet-style takeover very realistic, but a chernobyl-style "mistake" of a misinterpreted command, which results the unstoppable rampage of the fleet, doesn't seem to me impossible.
