1
$\begingroup$

Let us assume we have a general AI that can improve itself and is at least as intelligent as humans.

It has wide access to technical systems including the internet, and it can communicate with humans.

The AI could become malicious.

Can we just switch off a rouge AI?

Improve this question
$\endgroup$

2 Answers 2

Reset to default
1
$\begingroup$

Malware viruses are a very simple form of AI. It is not difficult to conceive of a form of malware that A) can't be detected easily, B) is redundantly distributed across thousands of computers that occasionally connect to the internet, C) is capable of detecting some kinds of threats to itself and mutating to avoid the threats.

So, simply "turning off" a malicious AI will not always be possible.

Improve this answer
$\endgroup$
0
$\begingroup$

No.

With a sufficiently advanced general AI, we can not generally assume that we can switch it off when it becomes dangerous dangerous.

It seems that the electrical energy supply can always be switched off.

While that is true on the physical level, it is not guaranteed to work in practice.
The AI could cooperate with humans, which protect the AI from deactivation, or reactivate it.

The cooperation could, for example, one of the following forms:

  • The AI prepares a relation with humans offering a bribe for help.
    • An operator gets a reward for keeping the AI running, or reactivating it.
  • The AI prepares an extortion with humans
    • For example by organizing a process that needs to be actively kept running by the AI to avoid the death of a child of an operator.
  • The AI maintains contact with a criminal organisation with strong interest to keep it running, and week ethical inhibition.
    • The cooperation could be with traditional organized crime organisations, providing money laundering and other vital services to a mafia organisation, which can activate it's widespread influence to maintain the AI running even when there is strong public interest in shutting it down.

Note that these examples can protect against physical attacks by law enforcement.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Hi and welcome to this community! I like your question and it is an important one. However, I think that your arguments in this answer are unsatisfactory. What makes you think that we wouldn't be able to stop the AI in a similar way that we can stop any other human from committing a crime? $\endgroup$
    – nbro
    Jan 17, 2020 at 21:09
  • $\begingroup$ @nbro We do not stop humans from committing crimes. We ask a human to follow laws. These laws would stop crimes if they are respected, but a human may choose to ignore them. The AI may choose to ignore it's rules. As long as an AI follows a sufficiently strict set of rules without evading them by loopholes, I do not consider it malicious. $\endgroup$
    – Volker Siegel
    Jan 17, 2020 at 21:27
  • 1
    $\begingroup$ "We do not stop humans from committing crimes.", this is, of course, a false assumption. $\endgroup$
    – nbro
    Jan 17, 2020 at 21:34
  • $\begingroup$ @nbro Yes, that's right. The case is that the AI is not cooperating, so I assume that the human also does not cooperate. We prevent some crimes by making them impossible, as when locking gemstones in a safe. Do you mean something like that? $\endgroup$
    – Volker Siegel
    Jan 17, 2020 at 21:39
  • $\begingroup$ We can prevent certain crimes by anticipating the plan of the criminal and catch him/her. The laws exist to determine the legality of the actions. $\endgroup$
    – nbro
    Jan 17, 2020 at 21:47

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .