-1
$\begingroup$

If a reinforcement learning algorithm uses a Deep Neural Network to predict the action given a state (a NN for a policy function), an Monte Carlo Tree Search in a model-based learning setup, then would a better accuracy of the policy function NN always indicate better model learning?

I mean accuracy, or other metrics like precision, recall etc, are always looked forward to, as being the metrics to assess model performance, then what's different in Reinforcement Learning?

$\endgroup$
-1
$\begingroup$

In supervised learning, the data that we have is real world data, and we want our model to mimic that input-output mapping as much as possible. For example, if we have MRIs of patients and we want to detect whether that patient has a cancer or not, then ofcourse we would want our model to predict every single patient correctly.

Now, come in the Reinforcement Learning domain. You have an agent, it acts in an environment, gets reward and another state, and sometime later the episode ends. The data on which the policy function neural network will be trained on, will the the data generated by the agent. And using the Monte Carlo Tree Search, the agent would pick the better action which he thinks is better. Now, the actions that the agent made at the corresponding states in one learning run, will become a training dataset for the other learning run. So, in the second learning run, if the agent is mimicing it's actions completely, then, well, it's not learning at all. It's just selecting the same actions over and over again, which kills the purpose of Reinforcement Learning.

Reward is the correct metric for Reinforcement Learning model assessment.

$\endgroup$
  • 2
    $\begingroup$ Reward is not the correct metric for RL. Expected Return might be. Although we support self-answers here, the question and answer need some work. They are difficult to understand, and it is hard to tell what the purpose is. You could start by trying explain what you mean by accuracy in the context of a policy function - the implication in the question is that you have some measure of correct action for the accuracy (presumably as returned during training from the MCTS?) $\endgroup$ – Neil Slater Sep 17 at 17:02
  • 1
    $\begingroup$ Further to that, expected return would not be a good metric in Alpha Zero - a recent system combining policy network and MCTS. Instead the Deep Mind team chose to use an estimated Elo rating metric. $\endgroup$ – Neil Slater Sep 17 at 17:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.