Return to Revisions

You don't need to wait until the end of an episode to receive rewards. At least, not in the general sense. If you have access to intermediate rewards, then you can update the value network sooner. But PPO actually uses the advantage function which is also an n-step rollout or done via Generalised Advantage Estimation which still relies on bootstrapping from estimates at different timesteps.