2
$\begingroup$

From what I have understood reading the UCT paper "Bandit based monte-carlo planning", MCTS/UCT requires a generative model.

  1. Does it mean, in case there is no generative model of the environment, we cannot use MCTS?
  2. If we can still use MCTS, how does the roll-out happen in this case, as there is no simulation?
$\endgroup$
  • $\begingroup$ It's unlikely anyone here has read the paper. But you could be lucky. $\endgroup$ – SmallChess May 9 '17 at 3:33
2
$\begingroup$

You either need a generative model or an emulator of the environment. In the later case you don't calculate your transitions and rewards using the model but feed your actions and states to the emulator and work with the results.

The emulator can be a black box as long as it returns the next state and the reward when provided with the current state and an action. You also need a way to identify all legal actions in a given state to build the tree.

$\endgroup$
  • $\begingroup$ thanks for the clarification; in my case, now I use the emulator of the environment $\endgroup$ – okkhoy Oct 31 '17 at 9:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.