In equation 4.9 of Sutton and Barto's book on page 79, we have(for policy iteration algo):
$\pi ^{'}(s) = arg \max_{a}\sum_{s',r}p(s',r|s,a)[r+\gamma v_{\pi}(s')]$
where $\pi$ is the previous policy and $\pi ^{'}$ is the new policy. Hence in iterations $k$ it must mean
$\pi _{k+1}(s) = arg \max_{a}\sum_{s',r}p(s',r|s,a)[r+\gamma v_{\pi_{k}}(s')]$
But in the example given in the same book on page 77 we have:
Now for the concerned state marked in red -
So $v_{\pi_{1}}$ = -1 for all four surrounding states
r = -1 for all four surrounding states
p(s',r|s,a) = 1 for all four surrounding states
$\pi _{2}(s) = arg \max_{a}[1*[-1+1*-1],1*[-1+1*-1],1*[-1+1*-1],1*[-1+1*-1]]$ $\pi _{2}(s) = arg \max_{a}(-2,-2,-2,-2)$
Hence this should give us a criss-cross symbol(4 directional arrow) in $\pi_{2}$(s) but here a left arrow symbol is given.
What's wrong with my calculations.