I've been following the Berkeley cs188's assignment (I'm not taking the course). Currently, they don't show the solution in the gradescope unless I get it correct.
My reasoning was
$V^*(a)$ = 10 fixed, because the optimal action is to terminate and receive the reward 10.
$V^*(b) = 10 \times 0.2 = 2$ using Bellman optimality eqn $V^*(s) = R(s)+ \gamma \ \rm{max}_{a} \sum_{s'} P(s'|s,a) V^*(s')$, where the optimal actionfrom b is to left.
Similarly, I get $V^*(c) = 10 \times (0.2)^2 = 0.4$
For the state $d$, it is optimal to move to the right and exit at $e$ to receive 1, therefore $V^*(d) = 1 \times 0.2 = 0.2$.
And $V^*(e) = 1$ fixed.
However, there autograder says it's incorrect and doesn't show explanation. Can anyone explain what the right approach or answer is?
