Return to Answer

The evaluation phase of an RL algorithm is the assessment of the quality of the learned policy and how much reward the agent obtains if it follows that policy. A typical metric that can be used to assess the quality of the policy is to plot the sum of all rewards received so far (i.e. cumulative reward or return) as a function of the number of steps. One RL algorithm dominates another if its plot is consistently above the other. You should note that the evaluation phase can actually occur during the training phase too. Moreover, you could also assess the generalization of your learned policy by evaluating it (as just described) in different (but similar) environments to the training environment [1].

• resources (CPU and memory) used, and/or
• experience/data/samples needed to converge to a certain level of performance (i.e. you can evaluate the data/sample efficiency of your RL algorithm).

The evaluation phase of an RL algorithm is the assessment of the quality of the learned policy and how much reward the agent obtains if it follows that policy. So, a typical metric that can be used to assess the quality of the policy is to plot the sum of all rewards received so far (i.e. cumulative reward or return) as a function of the number of steps. One RL algorithm dominates another if its plot is consistently above the other. You should note that the evaluation phase can actually occur during the training phase too. Moreover, you could also assess the generalization of your learned policy by evaluating it (as just described) in different (but similar) environments to the training environment [1].

• resources used (such as CPU and memory), and/or
• experience/data/samples needed to converge to a certain level of performance (i.e. you can evaluate the data/sample efficiency of your RL algorithm)
• robustness/sensitivity (i.e., how the RL algorithm behaves if you change certain hyper-parameters); this is also important because RL algorithms can be very sensitive (from my experience)

During learning (or training), you may not be able to find the optimal policy, so, when you want to use your learned policy to solve the actual real-world problem, should you strictly follow the action that gives you the highest reward at every state or maybe should you stochastically decide between actions? These are questions that you need to answer before deploying your RL algorithm.

The 12.6 Evaluating Reinforcement Learning Algorithms of the book Artificial Intelligence: Foundations of Computational Agents (2017) by David Poole and Alan Mackworth provides a section completely dedicated to the evaluation of reinforcement learning algorithms.

The evaluation phase of an RL algorithm is the assessment of the quality of the learned policy and how much reward the agent obtains if it follows that policy. A typical metric that can be used to assess the quality of the policy is to plot the sum of all rewards received so far as a function of the number of steps. One RL algorithm dominates another if its plot is consistently above the other. You should note that the evaluation phase can actually occur during the training phase too. Moreover, you could also assess the generalization of your learned policy by evaluating it (as just described) in different (but similar) environments to the training environment [1].

The linked section provides more details, so I suggest you read it.

During training, you want to find the policy. During the evaluation, you want to assess the quality of the learned policy. You can perform evaluation even during training.

During learning (or training), you may not be able to find the optimal policy, so how can you be sure that the learned policy to solve the actual real-world problem is good enough? This question needs to be answered, ideally before deploying your RL algorithm.

The evaluation phase of an RL algorithm is the assessment of the quality of the learned policy and how much reward the agent obtains if it follows that policy. A typical metric that can be used to assess the quality of the policy is to plot the sum of all rewards received so far (i.e. cumulative reward or return) as a function of the number of steps. One RL algorithm dominates another if its plot is consistently above the other. You should note that the evaluation phase can actually occur during the training phase too. Moreover, you could also assess the generalization of your learned policy by evaluating it (as just described) in different (but similar) environments to the training environment [1].

The section 12.6 Evaluating Reinforcement Learning Algorithms of the book Artificial Intelligence: Foundations of Computational Agents (2017) by Poole and Mackworth provides more details about the evaluation phase in reinforcement learning, so you should probably read it.

Apart from evaluating the learned policy, you can also evaluate your RL algorithm, in terms of

• resources (CPU and memory) used, and/or
• experience/data/samples needed to converge to a certain level of performance (i.e. you can evaluate the data/sample efficiency of your RL algorithm).

During training, you want to find the policy. During the evaluation, you want to assess the quality of the learned policy (or RL algorithm). You can perform the evaluation even during training.

The evaluation phase of an RL algorithm is the assessment of the quality of the learned policy and how much reward the agent obtains if it follows that policy. A typical metric that can be used to assess the quality of the policy is to plot the sum of all rewards received so far as a function of the number of steps. One RL algorithm dominates another if its plot is consistently above the other.

  You should note that the evaluation phase can actually occur during the training phase too.

The linked section provides more details, so I suggest you read it!

The evaluation phase of an RL algorithm is the assessment of the quality of the learned policy and how much reward the agent obtains if it follows that policy. A typical metric that can be used to assess the quality of the policy is to plot the sum of all rewards received so far as a function of the number of steps. One RL algorithm dominates another if its plot is consistently above the other. You should note that the evaluation phase can actually occur during the training phase too. Moreover, you could also assess the generalization of your learned policy by evaluating it (as just described) in different (but similar) environments to the training environment [1].

The linked section provides more details, so I suggest you read it.