Is the play of strong Chess AI easily distinguishable from human play?

Question

I don't play nearly enough Chess to be able to answer. For context, AlphaGo is stronger than the current strongest human player, but AlphaGo's game play has been cast as "inhuman" in the sense that it doesn't resemble human play. (In Go, this can involve aesthetic qualities.)

Really I'm wondering about "narrow" application of the Imitation Game/Turing Test, where one might design an automata to play more like a human, so that human players would be unable to determine if their opponent was a human or an automata.

Answer 1

AI programs that exist in today's world fall into the category of Narrow Intelligence. Narrow Intelligence are easy to distinguish when compared to General Intelligence (ones that resemble more like humans).
Highly advanced AI can often resemble to act like humans thought. I will like to talk about Deep Blue here.

Garry Kasparov, in a series of matches during Feb 1996, even though won three matches but, had a " shattering experience". Deep Blue flummoxed him in that first game by making a move with no immediate material advantage; nudging a pawn into a position where it could be easily captured. He said, "It was a wonderful and extremely human move" and this threw him for a loop. He further added that he had played a lot of computers but had never experienced anything like this. He added, "I could feel, I could smell, a new kind of intelligence across the table."
Next year he lost to an improved version of Deep Blue. Here also he lost mostly because of psychological reason. He later said that he was riled by a move the computer made that was so surprising and so un-machine-like that he was sure that IBM team had cheated. It was later found out that it was a glitch in Deep Blue. The program came across too many options and had no clear preference and thus played a random move.
It was this random move that shifted the game in favour of Deep Blue.

One advantage that Deep Blue (or other programs) have is the ability to perform far more computations in a given time. Humans, on the other hand, can extend their decision tree only to very limited depth. On having no clear decision to take, we take a random decision and try to learn from the experience, unlike most Narrow Intelligence programs present today. We need a fair amount of randomness and learning from experience to be added to the program in order to make it more like a human.

The answer to this question also depends on your position on "Chinese Room Argument". The argument holds that a program can't give a computer understanding or consciousness regardless of how intelligently or human like the program may make the computer behave. Chinese Room Argument focuses on something very important when it comes to "Consciousness v/s Simulation of Consciousness". John Searle argued there that it is possible for a machine (or human) to follow a huge number of predefined rules (algorithm), in order to complete the task, without thinking or possessing the mind.

Answer 2

There are three cases in which it is easily possible to distinguish strong AI play from the strong human play:

1. The AI is playing at superhuman skill level

This seems obvious, but I want to mention it for the sake of completeness. The current skill ceiling of top-level chess is well known and an opponent playing way above this skill ceiling must either be an AI or a chess guru who hid in a cave for the last centuries. Applying Occam's razor I would go with the AI. So to mimic human play the AI must make sure to stay at a human ELO level.

2. The AI plays an obvious weak move in a losing or strange situation

When the AI is in a bad situation and still tries to win the game without any reasonable move available, it might be tempted to play an obviously bad move, because the decision engine assigns it the highest probability of success. Such moves are considered rude at high-level play and a human player would not play them but resign. The most famous example of this is move 101 in the fourth game between Lee Sedol and AlphaGo. AlphaGo just lost a big group but still played T9 to extend it even further. This is a move a rookie player stops making after his first 10 games because it is that obvious that it does not work. A professional go player would never do this under normal circumstances. AlphaGo was out of options and played the rude move instead of resigning. This doesn't mean an AI cannot be programmed to avoid such behavior, but dire or simply very strange situations might induce moves like this, giving the nature of the AI away.

3. Other factors than the actual moves are considered

Depending on the test scenario, the AI might give away its nature through other means than the actual moves. Things like moving too fast or playing without breaks for hours. But I assume the question was just focused on the actual play and not the surrounding factors.

Those are the obvious examples where an AI might give away his true nature. I highly doubt that a strong AI, playing at the strong human level, can really be identified because of his "inhuman" style. I think it is more of a psychological pitfall to consider innovative moves from an AI "inhuman" and the same move from a pro player as "brilliant" if you know who is who. I am an amateur 3dan Go player - far from the top but with several years of experience under the belt - and at least I wasn't able to see anything "inhuman" in AlphaGos play so far. Ingo, it is well known that there are different styles of play, and after reviewing enough pro games one gets a sense of which player might have been educated in Korea, China, or Japan because certain patterns of strategies are more or less common in those countries. The lack of such an individual nuance might be an indication, that the player is actually a strong AI, but I am confident, that even the strongest pro wouldn't have a high success rate guessing who is human and who is the AI.

In chess, there are significantly fewer options at every move, many fixed patterns especially in the opening, and therefore less room for an individual style. Therefore I consider the same challenge even more difficult than in go. So if an AI has reason to "play like a human", this should be possible with the current level of AI technology.

Answer 3

I recall a friend saying that yes, it is somewhat obvious if you are playing against an AI.

From what he said, against normal players, there is a certain rhythm and structure that "makes sense". But AI play doesn't have this quality, "it doesn't make sense, but it just works". This seems to echo what you mentioned about the aesthetic quality of Go

Apparently because of this, chess/go AI can be detected during online play.

Answer 4

Yes - in chess the term "computer move" is used to denote a move found by a chess engine that a human player would never find (often because they make some slight improvement that a human would not be able to calculate). Humans use pattern recognition and some calculation in order to understand the chess position they are in while computers are able to calculate (~30 - 120+) moves ahead, a feat that a human player cannot match. It is for this reason that one can distinguish a human move from a computer move at some times.