Yet, I still have two important questions:
- How is the policy network updated? In (2), board positions are saved in a dataset of tuples (state, policy, value). The value is derived from the result of the self-played game. However, I’m not sure which policy is saved: the number of times that each move has been played, the prior probabilities for each move (I guess not) or something else?
- The cheat sheet says that (for competitive play) the move is chosen with the greatest N (=most visited). Wouldn’t it be more logical to choose the move with the highest probability calculated by the policy head?
Thanks in advance