In Example 4.3:Gambler's Problem of Sutton and Barto's book whose code is given here.
In this code the value function array is initialized as
states $\in[0,100]$ and the value function for optimal policy which is returned after solving it with value iteration is same as the one given in the book, but, if we only change the initialization of the value function in the code, suppose to
np.ones(states) then the optimal value function returned changes too, which means that the value iteration algorithm converges in both the cases but to different optimal value functions,but two different optimal value function is impossible in a MDP. So why is the value iteration algorithm not converging to optimal value function?
PS: If we change the initialization of value function array to
-1*np.random.rand(states), then the converged optimal value function also contains negative numbers which should be impossible as
rewards>=0, hence value iteration fails to converge to optimal value function.