My understanding of the main idea behind A2C / A3C is that we run small segments of an episode to estimate the return using a trainable value function to compensate for the unseen final steps of the episode.
While I can see how this could work in continuing tasks with relatively dense rewards, where you can still get some useful immediate rewards from a small experience segment, does this approach work for episodic tasks where the reward is only delivered at the end? For example, in a game where you only know if you win or lose at the end of the game, does it still make sense to use the A2C / A3C approach?
It's not clear to me how the algorithm could get any useful signal to learn anything if almost every experience segment has zero reward, except for the last one. This would not be a problem in a pure MC approach for example, except for the fact that we might need a lot of samples. However, it's not clear to me that arbitrarily truncating episode segments like in A2C / A3C is a good idea in this case.