I've been following the Berkeley cs188's assignment (I'm not taking the course). Currently, they don't show the solution in the gradescope unless I get it correct. enter image description here

My reasoning was

$V^*(a)$ = 10 fixed, because the optimal action is to terminate and receive the reward 10.

$V^*(b) = 10 \times 0.2 = 2$ using Bellman optimality eqn $V^*(s) = R(s)+ \gamma \ \rm{max}_{a} \sum_{s'} P(s'|s,a) V^*(s')$, where the optimal actionfrom b is to left.

Similarly, I get $V^*(c) = 10 \times (0.2)^2 = 0.4$

For the state $d$, it is optimal to move to the right and exit at $e$ to receive 1, therefore $V^*(d) = 1 \times 0.2 = 0.2$.

And $V^*(e) = 1$ fixed.

However, there autograder says it's incorrect and doesn't show explanation. Can anyone explain what the right approach or answer is?


1 Answer 1


Nevermind. I found that above answer is indeed correct, but the gradescope has a bug (it requires the format to be .2 instead of 0.2).


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .