How is the "right to explanation" reasonable?

Question

There has been recent uptick in interest in eXplainable Artificial Intelligence (XAI). Here is XAI's mission as stated on its DARPA page:

The Explainable AI (XAI) program aims to create a suite of machine learning techniques that:

• Produce more explainable models, while maintaining a high level of learning performance (prediction accuracy); and
• Enable human users to understand, appropriately trust, and effectively manage the emerging generation of artificially intelligent partners.

The New York Times article Can A.I. Be Taught to Explain Itself? does a good job at explaining the need for XAI from a human interest perspective as well as providing a glancing outlook on the techniques being developed for the same. The force behind XAI movement seems to center around the (up and coming?) concept of right to explanation, that is, the requirement that AI applications that significantly impact human lives via their decisions be able to explain to stakeholders the factors/reasons leading up to said decision.

How is the right to explanation reasonable, given the current standards at which we hold each other accountable?

Answer 1

How is the right to explanation reasonable, given the current standards at which we hold each other accountable?

In short, it is quite reasonable.

More specifically, making AI accountable and responsible for explaining the decision seems reasonable because

• Humans (DARPA in this case) has chosen to create, raise and evolve AI with the tax-payers money. In our society as well, whenever humans have come together for some purpose (some formal arrangement like government or otherwise), accountability and responsibility is assumed and assigned. Thus, expecting AI (who will take decisions on our behalf) to explain their decisions seems only natural extension of how humans currently operate.

• Humans (generally at least) don't have super-power or resources to control and manipulate the rest of the population. But when they do (like our political leaders), we want them to be accountable.

• In rare cases, powerful humans become powerful due to things which are accessible to everyone, while AI's super-power won't be. So, our civilization would feel safer and thus less insecure with an AI which doesn't shut the door on face when questions are asked.

Other benefits of AI that offers an explanation on how a decision is reached

• Far easier to debug and improve in its early stage.

• Can be customized further (and along the way) to amplify certain aspects like social-welfare over profitability, etc.

Answer 2

In the paper Slave to the Algorithm? Why a 'Right to an Explanation' Is Probably Not the Remedy You Are Looking For, the authors claim that the "right to explanation" is unlikely to provide a complete remedy to algorithmic harms for at least two reasons

1. It is unclear when any explanation-related right can be triggered

2. The explanations required by the law, "meaningful information about the logic of processing", might not be provided by current (explainable) AI methods

The authors then conclude that the "right to explanation" is distracting, but that other laws of the European Union's General Data Protection Regulation, such as

1. right to be forgotten
2. right to data portability
3. privacy by design

might compensate the defects of the "right to explanation".

To conclude, the "right to explanation" is an attempt to protect citizens against the possible undesirable consequences of the use of AI methods. However, it is not flawless and clearer rights might need to be promulgated.