I am reading Sutton and Barto's material now. I know value iteration, which is an iterative algorithm taking the maximum value of adjacent states, and policy iteration. But what is generalized policy iteration?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
In the standard policy iteration algorithm presented in Sutton and Barto's book, you alternate between a policy evaluation (PE) step and a policy improvement (PI) step (i.e. PE, PI, PE, PI, PE, PI, PE, ...). However, in general, you don't have to follow this alternation strictly in order to converge (in the limit) to the optimal policy. For example, value iteration (VI) is an example of a truncated policy iteration that still converges to the optimal policy.
The term generalized policy iteration (GPI) refers to all algorithms based on policy iteration, such as value iteration, that alternate in some order PI and PE, and that are guaranteed to converge to the optimal policy, provided PE and PI are executed enough times.
