3
$\begingroup$

I'm programming on Connect6 with MCTS.

Monte Carlo Tree Search is based on random moves. It counts up the number of wins in certain moves. (Whether it wins in 3 turns or 30 turns)

Is the move with less turns more powerful than the move with more turns?(as mcts just sees if it's win or not -- not considering the number of turns it took to win) And if so, is it meaningful to give bigger weight to the one with less turn win?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

Traditionally (when not considering your idea), the evaluation function for terminal game states would be implemented to return $1$, $0$, or $-1$ for wins, draws, or losses, respectively.

Changing that in a naive/straightforward way to make short-term wins more rewarding, long-term wins less rewarding, short-term losses more negative, and long-term losses less negative can be dangerous, it may change the objective that your agent is ultimately optimizing for (i.e. may lose the guarantee of converging towards optimal play given an infinite amount of time) if not done very carefully.

There is definitely value in considering the idea though, especially because in the Play-Out phase of MCTS, trajectories of (semi-)random moves introduce uncertainty in the evaluations at the end of those simulations, and this uncertainty increases as the length of the trajectories increases (due to increased number of uninformed decisions being made along the trajectory). Note that it is especially important to take into consideration here the number of moves played in the Play-Out phase, not necessarily including the number of moves made in the Selection phase (which are selected according to a much more informed strategy).

One paper I know of that investigates ideas along these lines is "Quality-based Rewards for Monte-Carlo Tree Search Simulations".

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .