# How can I derive n-step off-policy temporal difference formula?

I was reading the book "Reinforcement Learning: An Introduction" by Sutton and Barto. In section 7.3, they write the formula for n-step off-policy TD as

$$V(S_t) = V(S_{t-1}) + \alpha \rho_{t:t+n-1}[G_{t:t+n} - V(S_{t-1})],$$

where $$V(S_{t})$$ is state value function of the state $$S$$ at time $$t$$ and $$G_{t:t+n} \doteq \sum_{i=t}^{t+n-1}\gamma^{i-t}R_{i+1} + \gamma^n V(S_{t+n})$$ and $$\rho_{t:t+n-1}$$ is the importance sampling ratio.

I tried to prove this equation for $$n = 1$$ using the incremental update of the value function. Now I end up with this formula: $$V(S_t) = \frac{1}{t} \sum_{j=1}^{t} \rho_{j}G_{j}$$ $$V(S_t)= \frac{1}{t}(\rho_{t}G_{t} + \sum_{j=1}^{t-1}\rho_{j}G_{j})$$ $$V(S_t) = \frac{1}{t}(\rho_t G_t + (t-1)V(S_{t-1}))$$ $$V(S_t)=V(S_{t-1}) + \frac{1}{t}(\rho_{t}G_{t} - V(S_{t-1}))$$ I know I'm wrong because this does not match with the above equation. But can anyone please show me where I am wrong?

• Do you remember which version of the book you were reading at the time? – nbro Nov 5 '20 at 22:40
• It was a draft version – Swakshar Deb Nov 7 '20 at 14:26