I am trying to understand the value iteration method for Markov Decision Process(MDP) and I was referring ot UC Berkeley's slides titled Markov Decision Processes and Exact Solution Methods
On slide no.9, we start with the first step :
Ok! So, we have the information about the transition function (desribed elaborately in slide no.5 as well), the resting reward is 0 and discount of 0.9.
Using this, I am able to compute the utility value of the cell left to terminal state with R = +1 (Green cell). The action that is going to be most rewarding at this cell is moving forward, so putting the values in the equation as:
0.0 + 0.9(0.8*1 + 0.1*0 + 0.1*0) =0.72
Now, using the same algorithm I am able to compute the value of the cells adjacent to this newly obtained utility cell value. However, I really do not know how did they update the value from
0.72 -> 0.78
I have tried searching at various sites and seen some videos but most of them stop at first iteration assuming the next step is same as it is a recursive equation (And it should have been so!) but I am stuck at this!