I'm still struggling to convince myself. Maybe the following works: Since UCT converges to minimax, the probabilities $\pi$, extracted from MCTS, are almost optimal under our current value estimates (assuming we take enough steps). Now, in some sense $p$ is only as good as a single $v$. In other words, it's only as good as a single value estimate, while $\pi$ is close to optimal under value estimates for many many states... so it's probably better. This sounds half convincing, what do you think?