0
$\begingroup$

In the algorithm below, when $\tau + n \geq T$, shouldn't the algorithm bootstrap with the value of the next state? For instance, when $T=5, \tau=3, \& \; n=2$, we don't bootstrap the sample return with $V_{(\tau+n)}$, i.e., $V_5$ or the terminal state.

Text

Also, on line 4, what do we mean by "can take their index mod $n + 1$"?

Improve this question
$\endgroup$
0

1 Answer 1

Reset to default
1
$\begingroup$

Because the value of the terminal state is 0 by definition. There is no further reward to be obtained once you reach the terminal state.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .