The marvelous book Game Changer: AlphaZero's Groundbreaking Chess Strategies and the Promise of AI gave rise to this question. It is - in my opinion - a perfect example of rationalizing a piece of AI that makes it understandable and explainable. I favour the term "rationalizable" instead of "explainable" or "understandable" because it can naturally be applied to humans and animals as well (see rational animals), and because it captures better what it is (or should be) about: guessing the rationale behind some observed behaviour. A rationale lets an observed behaviour seem rational.

Candidates for rationizable AI are systems that show some sufficiently complex behaviour (not just giving classifying answers) that also humans may show. For example playing behaviour which is typically described in terms of strategies.

To give some examples how the authors of Game Changer rationalize AlphaZero (p. 98):

  • AlphaZero likes to target the opponent's king.

  • AlphaZero likes to keep its own king out of danger.

  • AlphaZero makes sure the central situation is stable before it weakens its own kingside structure to open lines against the opponent's king.

  • AlphaZero defends by creating confusion and introducing tactics into the game.

The authors even reconstruct AlphaZero's "thought processes" (see p. 77: "An in-depth illustration of one move")

To make such rationalizations you need a lot of expert knowledge. And AlphaZero most surely is not capable of giving such rationalizations in human-understandable form by itself.

My main question is for other specific examples of rationalizing (some piece of) AI.

Side questions are:

  • Do you find this kind of rationalization sound, and is it what one means when talking about explainable AI?

  • What has this concept of rationalizability to do with the game-theoretic concept of rationalizability?

Edit: Only today I found some references that explicitly mention "rationalization" or "rationality" in the context of AI:

This paper doesn't mention rationality explicitly but stresses on human-understandability. And it goes even one step further into the direction of self-explainability:

  • $\begingroup$ You're asking multiple questions here. I recommend that you edit your post to leave just one question or, at least, specify what is your main question (maybe you the other questions are just subquestions, but that doesn't seem to be the case). $\endgroup$
    – nbro
    Commented Feb 23, 2022 at 11:56
  • $\begingroup$ @nbro: Is it better now? $\endgroup$ Commented Feb 23, 2022 at 12:28


You must log in to answer this question.