In the book of Barto and Sutton, there are 3 methods presented that solve an RL problem: DP, Monte Carlo, and TD. But which category does policy gradient methods (or actor-only methods) classify in? Should I classify them as the 4th method of solving a reinforcement learning problem?
$\begingroup$
$\endgroup$
1
-
1$\begingroup$ What category do you consider that applies to DP, Monte Carlo and TD? You seem happy that these are categorised. For someone to answer, they will need to understand better how you are thinking about this. For instance, do you consider DP, Monte Carlo and TD to be categories of RL and you want to know which of these you should consider PG methods as? $\endgroup$– Neil SlaterJul 11, 2020 at 9:37
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
DP, Monte Carlo, and TD are methods of estimating returns. Policy gradient describes methods of learning a policy. So policy gradients serve a different purpose than the other things you mentioned. For clarity, you can use Monte Carlo or TD methods to estimate returns to construct the loss that you get your policy gradient from.
