I am trying to code a MCTS agent for tic tac toe and i have some theoretical questions regarding MCTS.

1)I am using the UCB1 MCTS $UCB(Si)=average value + 2*sqrt(ln(N)/ni)$ .

Considering the image below that describes a state diagram,i would like to ask if the UCB(s3)(for state S3) is equal to $3/3 + 2*sqrt(ln(6)/3)$ (i used the total number N of iterations of So ) OR it is $3/3 + 2*sqrt(ln(4)/3)$ (here i used the n1 total number of iterations of the parent node of s3,s4.

enter image description here

  1. I am wondering what's happening if during the expansion phase of the UCB1 MCTS algorithm i find a state that already exists in the tree. For example: Consider again the image (class diagram below) .During expansion of S1 i find again State s2 .What should i do in this case ? Should i treat s2 as i already visited it and copy its information or treat it like a completely new state ? And when i propagate it after the rollout should i update its stats everywhere?

enter image description here

3)I would like to ask specifically for tic-tac-toe what would be ideal number for the evaluation function.For example considering a ucb1 monte carlo UCB(si)$=average value + 2*sqrt(ln(N)/ni)$ what would be typical numbers for a rollout for a win , a lost and a draw for each player 'x','o'? Thank you in advance

  • $\begingroup$ You can use latex/mathjax on this site. Please, edit your post to use it. You just need to wrap your equations/formulas with $ on each side. $\endgroup$ – nbro Aug 9 at 10:54
  • $\begingroup$ To add subscripts, do like this: rather than $Si$, use $S_i$. Also, to put text in math mode use $\text{your_text_here}$. $\endgroup$ – nbro Aug 9 at 11:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.