6
$\begingroup$

Would it be ethical to allow an AI to make life-or-death medical decisions?

For instance, where there an insufficient number of ventilators during a respiratory pandemic, not every patient can have one. It seems like a straight forward question, but before you answer, consider:

  1. Human decision-making in this regard is a form of algorithm.

(For instance, the statistics and rules that determine who gets kidney transplants.)

  1. Even if the basis for the decision is statistical, the ultimate decision making process could be heuristic, so at least the bias could be identified.

In other words, the goal of this process, specifically, is favoring one patient over another, but doing so in the way that has the greatest utility.

  1. Statistical bias is a core problem of Machine Learning, but human decision making is also subject to this condition.

One of the arguments in favor might be that at least the algorithm would be impartial, here in relation to human bias.

Finally, where there is scarcity, utilitarianism becomes more imperative. (Part of the trolley problem is you only have two tracks.) But the trolley problem is also relevant because it can be a commentary on the burden of responsibility.

$\endgroup$
4
$\begingroup$

I will interpret the questions as being about triage. This is particularly important in crisis situations, where a lot of such life-or-death decisions have to be taken.

In the START system there are four different categories:

  • the deceased, who are beyond help
  • the injured who could be helped by immediate transportation
  • the injured with less severe injuries whose transport can be delayed
  • those with minor injuries not requiring urgent care.

According to the category assigned to a patient, you then decide what to do. Other systems might be more fine-grained, but the principle is the same: effectively the human decision is in the classification, which then guides the assignment of resources. In the above list, the second category would probably be the highest priority for treatment. But once the category has been decided, the course of action has been determined, though the actual treatment options will then be considered for each case.

The ethics are thus in the judgment of the survival chances: if a nurse decides patient X is too far gone to warrant treatment, that's it. So this is the hard part, the life-or-death decision.

There are then two aspects to consider:

  1. The accuracy of the diagnosis
  2. The predicted likelihood of treatment being effective

The diagnosis should be (I'm not a medic!) fairly neutral. There are of course mis-diagnoses, but there is no value judgment involved. There might be some 'leakage' between step 1 and step 2, in that a human being might be influenced by the prospects of treatment of a diagnosis when deciding what the problem is. As in: this is a nice person, I don't want them to die, so I (subconsciously) exclude the diagnosis X which would mean certain death.

In this case a computer system that had sufficient accuracy when making a diagnosis would IMHO actually be more ethical than a human being. Provided, of course, that there is no bias in the diagnosis. Which I think is a practical problem, but in theory could be dealt with.

Once the diagnosis has been determined, the estimation of treatment success is the next decision. This also can be subject to human error, and a computer system with access to statistical information (again, unbiased) of issues and their likelihood of survival could be more independent of other factors which a human being might use, but which are unrelated to the case.

So basically, I would say that a computer system could be more ethical than a human being in such a situation. Because you can defend the way it reached it's decision, and it has been without taking into account factors not related to the problem at hand.

However, it's not always that easy. There are plenty of cases where other factors influence the decision. As long as there are fixed rules, that might not be a problem, though. Some issues would be (in each case the patients would have the same diagnosis and projected survival chance):

  • One ventilator, but an 8-year-old child and a 85-year-old man
  • A 30-year-old pregnant woman and the Prime Minister of the country
  • A one-month-old baby and a 25-year-old student
  • A homeless person and a billionaire

As a given, the system would have to be agnostic of general features such as gender, race, religion, etc. But how do you take into account age, social status, and a whole host of other factors?

If there is no difference in the situation, the fairest way would be to toss a coin. That would surely upset a lot of people ("How can you justify leaving our president to die in this time of crisis!"), but if you had explicit rules ("If one patient is in a higher tax bracket, they get priority") you might upset even more people. The advantage of having such rules would be to make the bias explicit, and in a way protect a nurse or paramedic from having to decide between a rich senator and a homeless person — whose family is more likely to go after you if you decided against them? And if you have explicit, unambiguous rules, why not use them to guide an AI system?

Every human being has their own preferences, and I am glad that I don't have to make those kinds of decisions — it sounds like a horrific task to me. How can you even sleep in peace after basically having condemned one each of the above cases to death?

That is another factor about the ethics of AI: by relieving humans from having to make such decisions, it would also have a beneficial effect. If the final decision is the same that the human would come to, then it's a win/win situation. But that is probably unlikely, due to a whole range of subconscious biases.

The prime issue to me seems the lack of recourse in the case of "computer said no". When a human makes a decision (like a referee in a game), there will always be an argument if people are unhappy with it. But in this case there could be none. The oracle has spoken, your father will be left to die. Sorry, no other outcome possible. It would probably be the same with a human decision, but it wouldn't feel as 'cold': you can observe that the person making the decision has to struggle with it. And you might understand that it was not an easy choice. With a computer, that element is missing.

Anyway, to summarise: given various caveats,

  • sufficiently accurate and unbiased diagnosis,
  • unbiased prediction of treatment outcome,
  • transparent handling of non-medical factors,

I would say that an AI system would me more ethical, as it has a principled way of reaching a decision which does not disadvantage any group of patients and would always get the same decision for the same patient; furthermore it takes a heavy burden off triage staff who otherwise have to make such decisions.

Which does not mean that I would be happy with my loved-ones' survival being decided by such a system :)

| improve this answer | |
$\endgroup$
  • $\begingroup$ Great answer, and very well explicated! (My own answer focuses on only one of your points.) Re: heads of state and people with sufficient wealth, I suspect exceptions would be made, regardless of any ethical factor. $\endgroup$ – DukeZhou Apr 14 at 22:12
3
$\begingroup$

Oliver's answer is interesting and it provides valuable information (such as a brief description of the triage process, which I was not aware of), but I disagree with his conclusion or, at least, I think it can be misleading because he is implying it's "more ethical" because the AI will behave in "more principled way". It depends on your definition of "ethical" (and I will recall one below) and the implications of behaving in a "more principled way".

First of all, we should emphasize that current AI systems can be and usually are biased because they are mainly trained with data associated with humans and their actions (as pointed out by Gerry in his answer). Furthermore, currently, AI systems (including the ones for healthcare) are only designed by humans, who can automatically and often inadvertently introduce bias, for example, by choosing the specific AI model over another, the data, how to process or acquire the data. (Maybe, in the future, AI systems will design other AI systems, but can this really reduce bias? Given that humans will probably design the first AI system that is able to design other AI systems, would the bias introduced in this first AI system also be propagated to the other AI systems?)

In principle, an AI could make more rational decisions, especially if it is not affected by human limitations (which often are not limitations, such as feelings; e.g., if you didn't feel pain when hitting a chair, you would start bleeding without even not noticing it) that make humans sometimes take irrational actions.

However, is the rational action also the most appropriate one? It depends on your definition of rational action and what we mean by "appropriate one".

Here's a definition of "rational" from the dictionary

based on or in accordance with reason or logic

Although the AI system takes actions systematically by following the rules of logic, those actions are still based on some principles or axioms, which will bias the system. So, a rational agent can still be biased, but this bias will be systematic.

In general, every decision can potentially be biased because it's based on some principles and taken by a "subject".

Now, let's address your question more directly

Would it be ethical to allow an AI to make life-or-death medical decisions?

First, let me report two definitions of "ethical" from the dictionary

relating to moral principles or the branch of knowledge dealing with these:

morally good or correct

The original question can thus be rephrased as

Would it be morally good to allow an AI to make life-or-death medical decisions?

Of course, it's difficult to argue what is morally good or not, because this is often subjective. It's morally good for me to help my friends, but it isn't necessarily morally good to help other people. We have different friends, so this automatically implies that morally good is subjective.

The answer to this question ultimately boils down to the philosophical issue of good vs bad, which is naturally subjective. So, the answer to this question will depend on the philosophical ideas of each person. Some people will say "yes" and some people will say "no".

I think it's more productive to answer the question

What are the advantages and disadvantages of allowing an AI to make life-or-death medical decisions?

This question can be answered more objectively. For example, we could say that this would free humans from doing this job, which, in certain scenarios (as Oliver points out in his answer), can be "inconvenient". However, we could also say that current AI systems are still not compatible with human values and they do not think in the way humans do, so they could unexpectedly take "wrong" actions, which can also be difficult to explain (especially if your AI system is or uses a black box system, such as a neural network).

So, should AI systems be used to make life-or-death medical decisions?

I think that people should decide "democratically", and there's should be a great majority of acceptance, e.g. not just 51%, but e.g. 95-99% of people should agree with the idea of letting an artificial system to take a life-or-death decision. To take a reasonable vote, people should be aware of the consequences of such a vote, which means that people should be aware of the inner workings of the AI system and what they can or not do (which is often not possible when the AI system is also composed of black-box models, such as neural networks). Alternatively, allowing or not an AI to make such a decision can also be done on a case-by-case basis.

All these issues are related to "explainable artificial intelligence", "accountability", "transparency", which have been increasingly debated in the last years.

| improve this answer | |
$\endgroup$
3
$\begingroup$

At face value, this sound monstrous--a measure to offload responsibility to a non-conscious mechanism that cannot be meaningfully punished for mistakes.

However, I will argue:

  • There is humane benefit in taking this decision out of the hands of doctors re: the psychological toll

Specifically, doctors are not the reason for resource scarcity, yet they're the ones being forced to make scarcity-driven life-of-death decisions, and that has got to take a toll.

Essentially, unless one is a sociopath, there is going to be an emotional effect. Here the "sociopathy" of a pure algorithm relieves humans of this terrible burden.

(Might even reduce burnout, and keep more doctors working longer and with more focus.)

| improve this answer | |
$\endgroup$
3
$\begingroup$

I disagree with the idea that a trained Machine Learning model would be impartial. Models are trained on data sets that contain features. Humans prepare those data sets and decide what features are included in the data set. The model only knows what it is trained on. Human bias is still there just less blatantly obvious.

To address your question directly, the answer I believe is that it is no more or less ethical than to have humans make such decisions since in the end humans created the AI model.

My concern, however, is simply this:

Once we offload this to AI we will longer feel responsible for the results. The mentality of "the machine" made the choice will make it very easy to allow us to abdicate. This is especially true if the people implementing the AI decisions are not the ones who developed the AI. Of course, humans having to repeatedly make life and death will suffer a serious and potentially devastating toll. So, in the end, it is a trade-off, but, from my perspective, I think the risk of abdication and the consequences thereof carry a heavier weight. But then I am not the one faced with making life and death choices.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Actually, a good way to potentially strengthen this answer might be to attack my supposition that the final mechanism could be purely heuristic... $\endgroup$ – DukeZhou Apr 14 at 22:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.