# Why do we need to go back to policy evaluation after policy improvement if the policy is not stable?

Above is the algorithm for Policy Iteration from Sutton's RL book. So, step 2 actually looks like value iteration, and then, at step 3 (policy improvement), if the policy isn't stable it goes back to step 2.

I don't really understand this: it seems like, if you do step 2 to within a small $$\Delta$$, then your estimate of the value function should be pretty close to optimal for each state.

So, why would you need to visit it again after policy improvement?

It seems like policy improvement only improves the policy function, but that doesn't affect the value function, so I'm not sure why you'd need to go back to step 2 if the policy isn't stable.

Value functions are always specific to some policy, which is why you will often see the subscript $$\pi$$ in e.g. $$v_{\pi}(s)$$ when there is a defined policy.