# Monte Carlo Tree search UCB1 for Tic-Tac-Toe help

I am trying to code a MCTS agent for tic tac toe and i have some theoretical questions regarding MCTS.

1)I am using the UCB1 MCTS $$UCB(Si)=average value + 2*sqrt(ln(N)/ni)$$ .

Considering the image below that describes a state diagram,i would like to ask if the UCB(s3)(for state S3) is equal to $$3/3 + 2*sqrt(ln(6)/3)$$ (i used the total number N of iterations of So ) OR it is $$3/3 + 2*sqrt(ln(4)/3)$$ (here i used the n1 total number of iterations of the parent node of s3,s4.

1. I am wondering what's happening if during the expansion phase of the UCB1 MCTS algorithm i find a state that already exists in the tree. For example: Consider again the image (class diagram below) .During expansion of S1 i find again State s2 .What should i do in this case ? Should i treat s2 as i already visited it and copy its information or treat it like a completely new state ? And when i propagate it after the rollout should i update its stats everywhere?

3)I would like to ask specifically for tic-tac-toe what would be ideal number for the evaluation function.For example considering a ucb1 monte carlo UCB(si)$$=average value + 2*sqrt(ln(N)/ni)$$ what would be typical numbers for a rollout for a win , a lost and a draw for each player 'x','o'? Thank you in advance

• You can use latex/mathjax on this site. Please, edit your post to use it. You just need to wrap your equations/formulas with $on each side. – nbro Aug 9 at 10:54 • To add subscripts, do like this: rather than$Si$, use$S_i$. Also, to put text in math mode use$\text{your_text_here}\$. – nbro Aug 9 at 11:06