# Why a single trajectory can be used to update the policy network $\theta$ in A3C?

The Deep RL bootcamp on policy gradient techniques gives the update equation for the policy network in A3C as

$$\theta_{i+1} = \theta_i + \alpha \times 1/m \sum_{k=1}^m\sum_{t=0}^{H-1}\nabla_{\theta}log\pi_{\theta_i}(u_t^{(k)} | s_t^{(k)})(Q(s_t^{(k)},u_t^{(k)}) - V_{\Phi_i}^\pi(s_t^{(k)}))$$

However in the actual A3C paper, the gradient update is based on a single trajectory and there is no averaging of the gradient over $$m$$ trajectories as defined in the video ? The simple action-value actor-critic algorithm also does not seem to require an averaging over m trajectory.

I guess the gradient of the expectation of the Utility function, $$\nabla_{\theta}J(\theta)$$ in policy gradient methods where $$\nabla_{\theta}J(\theta) = E_{\tau \sim p(\tau ; \theta)}[r(\tau)\nabla_{\theta}log p(\tau;\theta)]$$ can be approximated using a single sample trajectory as shown in a deep reinforcement learning lecture by stanford where $$J(\theta) \approx \sum_{t > 0}r(\tau)\nabla_{\theta}log\pi_{\theta}(a_t |s_t)$$ and an average over sampled trajectories is not needed to compute the gradient for $$\theta$$ update in the direction of gradient.