I am implenting a Monte Carlo Tree Search algorithm, where the selection process is done through Upper Confidence Bound formula:
def uct(state): log_n = math.log(state.parent.sim_count) explore_term = self.exploration_weight * math.sqrt(log_n / state.sim_count) exploit_term = (state.win_count / state.sim_count) return exploit_term + explore_term
I have trouble however choosing the initial value for UCT, when the sim_count of the node is 0. I tried with +inf (which would be appropriate as approaching lim -> 0 from the positive side would give infinity), but that just means the algorithm will be always choosing an unexplored child.
What would you suggest as a initial value for the uct?
Thank you in advance!