# Calculating TD($\lambda$) returns, reinforcement learning

I am having some trouble with answering the following question:

A rat is involved in an experiment. It experiences one episode. At the first step it hears a bell. At the second step it sees a light. At the third step it both hears a bell and sees a light. It then receives some food, worth +1 reward, and the episode terminates on the fourth step. All other rewards were zero. The experiment is undiscounted ($$\gamma = 1$$). Approximate the state-value function by a linear combination of these features with two parameters: $$b · bell(s) + l · light(s)$$. If $$b = 2$$ and $$l = −2$$, write down the sequence of $$\lambda$$-returns $$v_t^\lambda$$ corresponding to this episode, for $$\lambda = 0.5$$.

$$v^\lambda_1 = 0.5 (−2 + 0.5×0 + 0.5×1) = −3/4$$

$$v^\lambda_2 = 0.5 (0 + 1×1) = 1/2$$

$$v^\lambda_3 = 0.5 (2×1) = 1$$

However, these are my calculations:

$$v^\lambda_1 = 0.5 (.5^0×−2 + .5^1×0 + .5^2×1) = 0.875$$

$$v^\lambda_2 = 0.5 (.5^0×0 + .5^1×1) = 1/4$$

$$v^\lambda_3 = 0.5 (.5^0×1) = 1/2$$

Any help is appreciated!

Edit: These are the functions used:

$$v^{(n)}_t = r_{t+1} +\gamma r_{t+2}+\dots+\gamma^{n−1} r_{t+n} + \gamma^n V_{(s_{t+n})}$$

$$v^\lambda _t = (1−\lambda)\sum_{n=1}^\infty \lambda^{n−1} v^{(n)}_t$$

bell state = $$[1 \ \ 0]^T$$

light state = $$[0 \ \ 1]^T$$

bell and light state = $$[1 \ \ 1]^T$$