People who code: we want your input. Take the Survey

# Tag Info

I think that this is an intentional subtle detail of the algorithm that ensures the convergence property. The claim in the book is that for any $b$ that provides us with "an infinite number of returns for each pair of state and action" the target policy $\pi$ will converge to optimal. Imagine now that we have such a bad policy $b$ that it never ...