# Tag Info

My understanding is that you want to extend the main update loop like this: $\qquad V(s) \leftarrow \text{max}_a \sum_{r,s'} p(r,s'|s,a)[r+\gamma V(s')]$ $\qquad \pi(s) \leftarrow \text{argmax}_a \sum_{r,s'} p(r,s'|s,a)[r+\gamma V(s')]$ Where the first line is same as original, the second line is taken from the end update that calculates the optimal policy....