I'm following along with PyTorch's example implementations (found here) of reinforcement learning algorithms that happen to be largely REINFORCE (vanilla policy gradient) based, and I notice they don't use batches. This leads me to ask, are batch updates of the network actually useful in this context?
Adding on, in my particular environment there's not a real meaningful cutoff for episodes as it's really set up for a sort of continuous play. As such, any n-length trajectory + rewards I collect is just as valid as another. For that reason, it would seem to mean that a longer episode/trajectory would serve the same purpose batches tend to in network updating.
Is it expected then that batches are not particularly worthwhile in the REINFORCE context, or is this just coincidence of the implementation I'm using? And is that answer amended if there are no meaningful episode cutoffs?