I searched a lot, but I was not able to find any tutorial, mostly because Gym environments and most RL libraries are not for multi-agent RL.
After checking the Internet, you will probably find several resources such as
- https://arxiv.org/abs/2011.00583 and others.
Try to understand the principles first (see above). After some reasonable amount of coding you can adapt OpenAI gym. Good luck!
Update 17 March 2022:
You may want to check this popular repository as well https://github.com/Farama-Foundation/PettingZoo
Multi-Agent Deep Reinforcement Learning in 13 Lines of Code Using PettingZoo
A tutorial on multi-agent deep reinforcement learning for beginners.
This tutorial provides a simple introduction to using multi-agent reinforcement learning, assuming a little experience in machine learning and knowledge of Python.