1
$\begingroup$

I know return is total discounted reward of an episode,so it is easy to plot return vs. episode.But in many papers,they provide figures of average return vs step,like this:enter image description here

Based on my knowledge,return is discounted cumulative rewards of an episode,it can't be calculated until the end of episodes,so I feel puzzled about how to calculate average return at each training step.

Does return of each training step mean discounted cumulative rewards collected from current training step to the end of current episode?If an agent is trained with N episodes,and each episode has T step,then I can calculate return of the (i-1)Tth step to the iTth step when the ith episode ends and plot return vs. step if all N episodes finish,is it right?

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ Please link the paper where you found the graphs. Answers can then explain those graphs with more confidence. The answers may also apply to other graphs you have seen, but you should always check the paper details to find what definitions are being used $\endgroup$
    – Neil Slater
    Sep 27 at 11:24
  • 1
    $\begingroup$ Here is the paper: arxiv.org/abs/2010.02966 $\endgroup$
    – waylone
    Sep 28 at 13:19

1 Answer 1

Reset to default
1
$\begingroup$

You have not linked the paper where you got such images, however, usually in DRL publications with these environments, the x-axis refers to training-steps, not environment-steps

Improve this answer
$\endgroup$
2
  • $\begingroup$ Here is the paper: arxiv.org/abs/2010.02966 $\endgroup$
    – waylone
    Sep 28 at 13:18
  • $\begingroup$ I'm new to DRL. If x axis refers to training steps,then how to calculate return at each training step?Should I use the policy learned at this training step to interact with the environment for a complete episode to get return of current training step? Thank you. $\endgroup$
    – waylone
    Oct 6 at 6:24

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .