I was reading the book "Reinforcement Learning: An Introduction" by Sutton and Barto. In section 7.3, they write the formula for n-step off-policy TD as
$$V(S_t) = V(S_{t-1}) + \alpha \rho_{t:t+n-1}[G_{t:t+n} - V(S_{t-1})],$$
where $V(S_{t})$ is state value function of the state $S$ at time $t$ and $ G_{t:t+n} \doteq \sum_{i=t}^{t+n-1}\gamma^{i-t}R_{i+1} + \gamma^n V(S_{t+n})$ and $\rho_{t:t+n-1}$ is the importance sampling ratio.
I tried to prove this equation for $n = 1$ using the incremental update of the value function. Now I end up with this formula: $$V(S_t) = \frac{1}{t} \sum_{j=1}^{t} \rho_{j}G_{j} $$ $$V(S_t)= \frac{1}{t}(\rho_{t}G_{t} + \sum_{j=1}^{t-1}\rho_{j}G_{j}) $$ $$V(S_t) = \frac{1}{t}(\rho_t G_t + (t-1)V(S_{t-1}))$$ $$V(S_t)=V(S_{t-1}) + \frac{1}{t}(\rho_{t}G_{t} - V(S_{t-1}))$$ I know I'm wrong because this does not match with the above equation. But can anyone please show me where I am wrong?
