0
$\begingroup$

In the algorithm below, when $\tau + n \geq T$, shouldn't the algorithm bootstrap with the value of the next state? For instance, when $T=5, \tau=3, \& \; n=2$, we don't bootstrap the sample return with $V_{(\tau+n)}$, i.e., $V_5$ or the terminal state.

Text

Also, on line 4, what do we mean by "can take their index mod $n + 1$"?

$\endgroup$
0
1
$\begingroup$

Because the value of the terminal state is 0 by definition. There is no further reward to be obtained once you reach the terminal state.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.