# How does n-step Temporal Difference remove the notion of time-step?

How does n-step TD removes the notion of time-step as referenced in Sutton and Barto (2nd edition, Page 163) below?

Another way of looking at the benefits of n-step methods is that they free you from the tyranny of the time step. With one-step TD methods the same time step determines how often the action can be changed and the time interval over which bootstrapping is done. With one-step TD methods, these time intervals are the same, and so a compromise must be made. n-step methods enable bootstrapping to occur over multiple steps, freeing us from the tyranny of the single time step.

Consider the following example.

We know our n-step update equation as: $$V_{t+n}(S_t) = V_{t+n-1}(S_t) + \alpha [G_{t:t+n} - V_{t+n-1}(S_t)]$$

Now, let $$t=0$$ and $$n=2$$. This gives us: $$V_2(S_0) = V_1(S_0) + \alpha [G_{0:2} - V_1(S_0)]$$.

Before our n-step TD prediction algorithm starts, we initialize with $$V_0$$. But we use $$V_1$$. Why? And how do we calculate $$V_1$$?

• It seems to me you're asking 2 distinct questions here. "how do we calculate V1?" and "how does n-step TD removes the notion of time-step?". Please, edit your post to leave just one question per post, although the questions are related to the same topic. Ask the second question in a different post.
– nbro
Aug 21 at 10:41
• Actually, I wanted to use the former question as an example to illustrate actual question I would like to be answered. I edited the question again. If the issue still persists, I'll split the questions into different post. Aug 21 at 17:20