A human player plays limited games compared to a system that undergoes millions of iterations. Is it really fair to compare AlphaGo with the world #1 player when we know experience increases with the increase in number of games played?
$\begingroup$
$\endgroup$
1
-
$\begingroup$ Define fair. :) $\endgroup$– JimFeb 3, 2018 at 15:51
Add a comment
|
5 Answers
$\begingroup$
$\endgroup$
Is it fair to compare AlphaGo with a Human player?
Depends on the purpose of the comparison.
If we are comparing ability to win a game of Go, then yes.
If we are comparing learning ability, then maybe. It depends on the task. AlphaGo and systems like it are capable of learning only in well-described limited domains. There may be an analogy with sensory learning (it might even be possible in theory to take a small piece of brain tissue and run an algorithm similar to AlphaGo's learning process on it).
In general, the approach used by AlphaGo and other reinforcement learning successes is "trial-and-error plus function approximation". It seems analogous to perception and motor skills, such as object recognition or riding a bike, as opposed to reasoning skills and games as humans play them, which goes through many more cognitive and conscious layers that have no real analog in a RL system like AlphaGo.
A human player plays limited games compared to a system that undergoes millions of iterations
This is an advantage of a machine to learn this kind of task. It would equally apply in other simulated environments with simple rules. If your goal is to have the most skilled and optimal navigation of such a domain, the implication now is that you would not train a human expert through years of study, but to write the simulator and train an AlphaGo-like machine.
This is no different a comparison than deciding cars and roads are better solutions to long distance travel for the general population than walking or horses and carts. It doesn't matter what underlies the advantage of one over the other, the assessment is cost/benefit, which resolves to a single comparable number.
It would, however, be wrong to assess AlphaGo as a better general-purpose learning engine than a human. The fact that humans do not have to work fully through millions of simulations in full detail is important. It means that something about how humans learn is still not covered by learning machines. Some of these things are understood and being discussed - such as the ability to focus intuitively on important aspects of what to learn, the ability to reason about the environment, learning analogously or transfer learning from other domains.
answered Jan 28, 2018 at 9:48
$\begingroup$
$\endgroup$
Yes it is. If we ever compare computers to humans we should take into acount the fact that computers can work 24 hours a day every day and faster than humans. That is bigest advantage of computers over humans.
$\begingroup$
$\endgroup$
If you read through the abstracts of Chess AI papers, it is often pointed out that humans "search" the Chess game tree much more efficiently than computers, which was why it was so hard to beat the top humans in Chess for so many years. (The human efficiency may have to do with intuition and judgement, which are difficult to replicate. "Confidence levels" for AI evaluations is one method of addressing these issues, as is "monte carlo". But it's also important to note that humans are far more limited in the depth and breadth of their "searches", which is why, now that we have the right algorithms, humans can no longer win.)
Is it fair?
Perhaps the more salient question is:
Is it useful to compare AlphaGo to a human player?
It most certainly is, because it tells us that we have is sometimes termed a "strong-narrow AI" that can outperform a human in a single task.
Why AlphaGo beating Lee Sedol was a big deal is the complexity of Go, the intractability of the Go game tree, and the fact that computers were previously ineffective against high-level human Go players.
This human vs. AI evaluation doesn't strictly fall under the "Turing Test" (Imitation Game), it does fall squarely under the maxim of Protagoras that "Man is the measure of all things."
This is critical because intelligence is a spectrum, and gauging strength of intelligence, in the context of intractable problems (problems that cannot be fully solved due to their size) is a function of relative strength of two agents, whether human or AI.
This relative assessment is all we have, and all we may ever have for certain sets of problems.
The problem with humans is not that we're not clever, but that our minds have cognitive limitations. So to tackle certain problems, intelligent machines are useful.
$\begingroup$
$\endgroup$
There is no such thing as fairness when comparing. You define a measure for performance and then compare the values of the measure.
One sensible measure for playing the game of GO is the 'Number of games won', regardless of any investment in the development of the system, computational or sample efficiency. AlphaGo is currently at the top by this measure.
Another sensible measure could be 'Number of games won under a restriction on sample efficiency during training'. As others pointed out, such a measure could be much more favorable for humans.
$\begingroup$
$\endgroup$
As a chess player and a AI/ML engineer, I can say yes, why not. I'm not sure why it isn't fair to compare anything if you give each side it's just due and do a 'fair comparison'. Obviously, what that encompasses is extremely subjective, but there are philosophical and logical measures of fairness.
Now speaking on the comparison, AlphaZero and a Human's learning styles are much more similar than that of a Human's and Stockfish. This is mainly due to the fact that human's in some capacity use RL, mainly in the dopaminergic neural pathways. While human behavior can certainly be modeled as an alpha/beta tree-search, it is not anything like the way we make decisions.
As far as the top humans, who cares? We we've been worse than computers for years.
