# How to compute the Retrace target for multi-step off-policy Reinforcement Learning?

I am implementing the A3C algorithm and I want to add off-policy training using Retrace but I am having some trouble understanding how to compute the retrace target. Retrace is used in combination with A3C for example in the Reactor.

I often see the retrace update written as

$$\begin{equation} \Delta Q(s, a) = \sum_{t' = t}^{T} \gamma^{t'-t}\left(\prod_{j=t+1}^{t'}c_j\right) \delta_{t'} \end{equation}$$

with $$\delta_{t'} = r(s_{t'}, a_{t'}) + \gamma \mathbb{E}[Q(s_{t'+1}, a_{t'+1})] - Q(s_{t'}, a_{t'})$$ and $$c_j$$ being the Retrace factors $$c_j = \lambda \min(c, \frac{\pi(a_j|s_j)}{b(a_j|s_j)})$$.

Now, when employing neural networks to approximate $$Q_{\theta}(s, a)$$ it is often easier to define a loss $$\begin{equation} \mathcal{L}_{\theta} = \left(G_t - Q(s, a)\right)^2 \end{equation}$$ and let the backward function and the optimizer do the update. How can I write the Retrace target $$G_t$$ to use in such a setup?

Is it correct to write it as follows? $$\begin{equation} G_t = \sum_{t'=t}^T \gamma^{t'-t} \left(\prod_{j=t+1}^{t'}c_j\right) (r_{t'} + \gamma Q(s_{t'+1}, a_{t'+1}) - Q(s_{t'}, a_{t'})) \end{equation}$$

and then compute $$\mathcal{L}$$ as above, take the gradient $$\nabla\mathcal{L}_{\theta}$$ and perform the update step $$Q(s_t, a_t) = Q(s_t, a_t) + \alpha \nabla\mathcal{L}_{\theta}$$ ?