I am running a MCTS as part of a game ML algorithm. The game that it is playing can reach the same state via multiple paths. It can also reach states again. This is similar to how a position on a chess board can be reached multiple ways.
When I am doing the MCTS, sometimes it gets stuck in a loop where it wants to run the same moves over and return to a previously visited state. So, like chess, I decided to add draw by repetition. However, this seems to leave me with two options:
- Drastically expand the Nodes of the MCTS and increase the state space. This is because the node is no longer defined by the game state and player turn alone. It also matters the path that was taken to it.
- Simply "short circuit" the MCTS process itself. When it reaches the same state (as defined by board/player turn) n times, return a value of 0 for simulation. I worry this will backfill along the tree and have unintended consequences. It also seems like bad practice to pollute the ML part with the rules of the game.
Are there any other solutions I'm missing? Does anyone know how folks handle it for chess?