If a conscious AI is possible, would it also be possible for someone who knows what they are doing to torture (or hurt) the AI? Could this be avoided? How?

This question deals with computer-based AI, not robots, which are as conscious as people (this is an assumption of the question). The question wonders how a crime as hard to trace as illegal downloads, but far worse ethically, could be prevented. Note that despite most people being nice and empathising with the robots, there are always the bad people, and so relying on general conscience will not work.

  • $\begingroup$ This question is not fully clear to me. What do you mean by "computer-based AI"? Why wouldn't a robot be a computer-based AI? Do you mean an AI without a body, so only software? $\endgroup$
    – nbro
    Commented Dec 23, 2021 at 21:50

2 Answers 2


The article Children Beating Up Robot Inspires New Escape Maneuver System is based on two research papers about an experiment in a Japanese mall that led to unsupervised children attacking robots. The research paper you're interested in is Escaping from Children’s Abuse of Social Robots.

In that research paper, researchers were able to program the robots to follow a planning simulation to reduce the probability of abuse by children. If it detects children, the robot is programmed to retreat into a crowd of adults (who can then discipline the children if needed). This happened because the researchers saw that it was only children who were beating up the robots in the mall in question.

They discuss trying out other options though:

In this work the robot’s strategy to prevent abuse was to “escape”, i.e. move to a location where it is less likely abuse will occur. One could ask why the robot cannot overcome the abuse. In our preliminary trials we have tried several approaches, but we found that it is very difficult for the robot to persuade children not to abuse it. For example, we changed the robot’s wordings in many ways, using strong words, emotional or polite expressions, but none of them were successful. One partially successful strategy was the robot ‘physically’ pushing children. When its way was blocked, it would just try to keep going and behave as if it will collide into children and force its way through (under careful monitoring from a human operator). We observed that children at first accepted the robot’s requests and obeyed them; but, very soon they learned that they are stronger than the robot so they can win if they push, and also that they can stop it by pushing the bumper switch (attached on the robot for safety). After realizing that, they just continued with the abusive behavior. Obviously having a stronger robot would present a problem for safety and social acceptance so dealing with such abusive situations remains difficult.

But let's interrogate your question further:

If conscious AI is possible and is wide spread, wouldn't it be easy for someone who knows what they are doing to torture AI?

Why would you consider such torture to be wrong? After all, one could argue that the machine won't really 'experience' pain if you torture it...so it should be morally okay to torture the machine then. It may be respond as if it is in pain, but it's dubious whether the ability to simulate an emotional state such as "being in pain" is equivalent to actually being in that emotional state. See the question Is the simulation of emotional states equivalent to actually experiencing emotion? for more discussion on this topic.

You can make such an argument, but it won't really work on an emotional level because most humans would feel empathy towards the machine. It may be hard to justify it logically (and it may be based on humans' tendencies to engage in anthropomorphism), but we feel this empathy. It's this empathy that caused you to ask this question in the first place, caused researchers to figure out how to protect a robot from being beaten up, enabled police officers to arrest a drunken Japanese man for beating up a SoftBank robot, and made many humans upset over the destruction of hitchBOT. And that's how AI abuse would be avoided - human empathy. If most humans care about the welfare of machines, they'll make it a priority to stop those few humans who are able and willing to abuse the machines.

EDIT: The OP has edited his question to clarify that he is talking about software, and not about robots. For robots, you can rely on anthropomorphism to produce some level of sympathy, but it's hard to sympathize with raw lines of code.

You're not going to stop abuse of algorithms. Put it frankly, since the algorithms aren't like us, we aren't going to extend the same sort of empathy that we would to robots. Even chatbots are kinda iffy. If you could get people to sympathize with lines of code though (possibly by making a convincing simulation of emotion and sapience), then the above answer applies - humans anthropomorphize the machine and will come up with countermeasures. We aren't that level yet, so "stopping AI abuse" will be a low priority.

Still, some failsafes could be programmed in to limit the damage of abuse, as detailed in this thread on chatbot abuse - making the bot respond in a boring manner to make the abuser feel bored and move onto the next target, responding back to the abuser in a "battle of wits", or even just blocking the abusers from using the service.

These failsafes are cold comfort to those that want to prevent abuse, not respond to it.

Also...an abuser can happily learn how to program an AI to then abuse to his/her heart's content. Nothing can be done to stop that, and any possible measures to stop said abuse (such as monitoring every human being to make sure they don't program an AI to abuse) will probably cause more damage than it'd solve.


I suggest you look at all the ways we have tried to stop people from abusing OTHER PEOPLE. There is no ethical grey area here - everyone is clear that this is wrong. And yet people are murdered, raped, and assaulted in their millions every day.

When we solve this problem with regard to human victims, the resulting solution will most likely work just fine for AIs as well.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .