Can AlphaZero considered as Multi-Agent Deep Reinforcement Learning?
I could not find a clear answer on this. I would say yes it is Multi Agent Learning, as there are two Agents playing against each other.
1 Answer
$\begingroup$
$\endgroup$
2
Depends on perspective.
On one hand, you have an agent playing in an environment with another agent also evolving. This falls under the definition of Multi-Agent Learning, as can be seen with works such as
Michael Bowling and Manuela Veloso. Multiagent learning using a variable learning rate. Artificial Intelligence, 136(2):215 – 250, 2002.
Michael Bowling. Convergence and no-regret in multiagent learning. In Proceedings of the 17th International Conference on Neural Information Processing Systems, NIPS’04, pages 209–216, Cambridge, MA, USA, 2004. MIT Press.
M. D. Awheda and H. M. Schwartz. Exponential moving average q-learning algorithm. In 2013 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), pages 31–38, April 2013.
Sherief Abdallah and Victor Lesser. A multiagent reinforcement learning algorithm with non-linear dynamics. Journal of Artificial Intelligence Research, 33:521–549, 2008.
However, you can also claim that you simply have a single agent learning on a non-stationary environment (the environment contains both the game rules and the opponent), and you simply learn on that basis. From this perspective, there is no multi-agent learning at all.
-
$\begingroup$ Thanks for your answer. But could not one argue with the second argument, that all multi agent learning is just single agent learning, as long as the agents do not need to act cooperatively? I mean it's just an agent in an non stationary environment? $\endgroup$– flexwCommented Aug 7, 2020 at 21:41
-
1$\begingroup$ Yes, you could argue that any "fully competitive environment" can be considered a non-stationary single. Tbh I don't like this interpretation, it was a discussion I had during my PhD defense, but it is still a PoV $\endgroup$ Commented Aug 8, 2020 at 0:01