I have a theoretical question about RL. I use Unity ml-agents, but maybe this is more a generalized thing.

In Unity ml-agents there is this concept that actions in episodes that generate more overall rewards will be prioritized over actions in less reward-full episode. This kind of doesn't make sense to me, because some action in the more rewarding episode can be worse then ones in the less rewarding one (especially for long episodes). And in general an action should prioritized depending on the step-to-step reward in respect to future rewards and those discount factor (gamma), right?

But why then giving tiny negative constant reward on every step can make the agent learn to end the episode faster? And why its recommended to have constant max step length for continuous tasks? And this concept of "agent comparing episodes" people are using?

Practical example is training a car to drive different length tracks. Longer track means more rewards. But should it create a problem, because of the "episode reward comparing"?

  • 1
    $\begingroup$ Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. $\endgroup$
    – Community Bot
    Apr 12, 2023 at 15:35
  • $\begingroup$ Instead of writing "Theoretical question about reinforcement learning episodes", just put your specific question in the title. $\endgroup$
    – nbro
    Apr 13, 2023 at 20:11


You must log in to answer this question.

Browse other questions tagged .