I am trying to understand the value iteration method for Markov Decision Process(MDP) and I was referring ot UC Berkeley's slides titled Markov Decision Processes and Exact Solution Methods
Ok! So, we have the information about the transition function (desribed elaborately in slide no.5 as well), the resting reward is 0 and discount of 0.9.
Using this, I am able to compute the utility value of the cell left to terminal state with R = +1 (Green cell). The action that is going to be most rewarding at this cell is moving forward, so putting the values in the equation as:
0.0 + 0.9(0.8*1 + 0.1*0 + 0.1*0) =0.72
Now, using the same algorithm I am able to compute the value of the cells adjacent to this newly obtained utility cell value. However, I really do not know how did they update the value from
0.72 -> 0.78
I have tried searching at various sites and seen some videos but most of them stop at first iteration assuming the next step is same as it is a recursive equation (And it should have been so!) but I am stuck at this!