To make A2C into A3C you make it asynchronous. From what I understand the 'correct' way to do that is to thread off workers with a copy of the policy and critic, and then return the state/action/reward tuples to the main thread, which then performs the gradients updates on the main policy and critic, and then repeat the process.
I understand why the copying would be necessary in a distributed environment, but if I were to always run it locally then could I just perform the updates on a global variable of the policy and critic, i.e. avoid the need for copying? Provided the concurrency of the updates was handled correctly, would that be fine?