5
$\begingroup$

According to Wikipedia Artificial general intelligence(AGI)

Artificial general intelligence (AGI) is the intelligence of a (hypothetical) machine that could successfully perform any intellectual task that a human being can.

According to the below image (a screenshot of a picture from When will computer hardware match the human brain? (1998) by Hans Moravec, and Kurzweill also uses this diagram in his book The Singularity Is Near: When Humans Transcend Biology), today's artificial intelligence is the same as that of lizards.

enter image description here

Let's assume that within 10-20 years, we, humans, are successful in creating an AGI, that is, an AI with human-level intelligence and emotions.

At that point, could we destroy an AGI without its consent? Would this be considered murder?

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ the picture is misleading: having large amounts of compute is not equivalent to being "intelligent" $\endgroup$
    – k.c. sayz 'k.c sayz'
    Mar 10, 2020 at 1:10

4 Answers 4

Reset to default
6
$\begingroup$

Firstly, an AGI could conceivably exhibit all of the observable properties of intelligence without being conscious. Although that may seem counter-intuitive, at present we have no physical theory that allows us to detect consciousness (philosophically speaking, a 'Zombie' is indistinguishable from a non-Zombie - see the writing of Daniel Dennett and David Chalmers for more on this). Destroying a non-conscious entity has the same moral cost as destroying a chair.

Also, note that 'destroy' doesn't necessarily mean the same for entities with persistent substrate, i.e. their 'brain state' can be reversibly serialized to some other storage medium and/or multiple copies of them can co-exist. So if by 'destroy' we simply mean 'switch off', then an AGI might conceivably be reassured of a subsequent re-awakening. Douglas Hofstadter gives an interesting description of such an 'episodic consciousness' in "A Conversation with Einstein's Brain"

If by 'destroy', we mean 'irrevocably erase with no chance of re-awakening', then (unless we have a physical test which proves it is not conscious) destroying an entity with a seemingly human-level awareness is clearly morally tantamount to murder. To believe otherwise would be substrate-ist - a moral stance which may one day be seen as antiquated as racism.

Improve this answer
$\endgroup$
3
  • $\begingroup$ pzombies are kind of a debated thing though; it's rather weird to say that you can know that an AGI has no consciousness if, following Chalmers words, there is no difference in the actions taken by zombies and people; if it is unethical to kill other people, but you cannot verify if they are pzombie or person, how is it ethical to kill an AGI? It seems you come to the same conclusion in your answer though, after rereading, so I'm gonna upvote. $\endgroup$
    – Pimgd
    Aug 4, 2016 at 12:16
  • 1
    $\begingroup$ @Pimgd - We're actually in agreement (like Dennett, I happen to think that Zombies are logically impossible). If an AGI appears indistinguishable from a human, it would be unethical to kill it unless you can prove it has no consciousness. No-one knows what such a proof might look like, or even if one exists. $\endgroup$
    – NietzscheanAI
    Aug 4, 2016 at 15:10
  • 1
    $\begingroup$ Great answer, although I will posit that the act of destruction, even with an non-conscious entity, is not necessarily neutral from a subjective perspective. I thinking about ideas like "intention" in a Buddhist context. I find that, as I get older and, ostensibly, become more "evolved", I have become more and more reluctant to rack up civilian casualties in games like Grand Theft Auto. Is this is merely a form of anthropomorphism, or is there some underlying, moral imperative? $\endgroup$
    – DukeZhou
    Dec 2, 2016 at 22:34
1
$\begingroup$

Even if machines with true Artificial General Intelligence were created, their apparent intelligence would still be by definition artificial. The word simulation is a synonym and could be used to redefine AGI as Simulated General Intelligence.

Keeping that in mind, a machine that appears to be expressing emotions would only be the result of a series of complicated algorithms allowing a computer to assess the situation and respond in an intellectually appropriate manner based on external stimulus and conditions. Every possible action this machine could possibly make would be derived from a list of possible actions the machine is capable of, no matter how large the number of possible actions grows. The machine is still a series sensors, programmed instructions, and cycles of execution.

Destroying such a machine could potentially be the destruction of property if it wasn't owned by the person who destroyed it, but would it be murder? No.

A broken machine can potentially be rebuilt and reactivated if it is broken. It never really died; it was destroyed. A living being that is killed is really dead and cannot be rebuilt and made alive once again. These key differences lead me to agree with the previous answer and conclude that no, destroying an artificial intelligence without its consent would not be murder.

Improve this answer
$\endgroup$
0
$\begingroup$

There is another problem besides consciousness, which is personal identity.

Consider an AI whose task is learning the discourse of a particular person during years, so as to simulate their conversation as closely as possible. After the death of the seed human, their artificial continuation can keep chatting with other humans (especially family members) using the ideas taught by the now dead person.

There is a strong link between such an intelligence and a human being, so it poses a problem harder than deleting a general chatbot with no individual attachment.

Improve this answer
$\endgroup$
0
$\begingroup$

Yes we can, just like we sentence living human to death. Moral or not depends on the particular time point we are in history.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .