I am trying to understand the value iteration method for Markov Decision Process(MDP) and I was referring to UC Berkeley's slides titled Markov Decision Processes and Exact Solution Methods
On slide no. 9, we start with the first step :
Ok! So, we have the information about the transition function (described elaborately in slide no. 5 as well), the resting reward is 0 and discount of 0.9.
Using this, I am able to compute the utility value of the cell left to terminal state with R = +1 (Green cell). The action that is going to be most rewarding at this cell is moving forward, so putting the values in the equation as:
$$0.0 + 0.9 (0.8*1 + 0.1*0 + 0.1*0) =0.72$$
which seems to be correct:
Now, using the same algorithm I am able to compute the value of the cells adjacent to this newly obtained utility cell value. However, I really do not know how did they update the value from
0.72 -> 0.78
in the next slide:
I have tried searching at various sites and seen some videos but most of them stop at the first iteration assuming the next step is the same, as it is a recursive equation (And it should have been so!), but I am stuck at this!