I have been trying to understand why MCTS is very important to the performance of RL agents, and the best description I found was from the paper Bootstrapping from Game Tree Search stating:
Deterministic, two-player games such as chess provide an ideal test-bed for search bootstrapping. The intricate tactics require a significant level of search to provide an accurate position evaluation; learning without search has produced little success in these domains.
I however don't understand why this is the case, and why value based methods are unable to achieve similar performance.
So my question would be:
- What are the main advantages of incorporating search based algorithms with value based methods?