7
$\begingroup$

I am reading Sutton and Barto's book on reinforcement learning. I thought that reward and return were the same things.

However, in Section 5.6 of the book, 3rd line, first paragraph, it is written:

Whereas in Chapter 2 we averaged rewards, in Monte Carlo methods we average returns.

What does it mean? Are rewards and returns different things?

$\endgroup$

2 Answers 2

6
$\begingroup$

Return refers to the total discounted reward, starting from the current timestep.

$\endgroup$
3
$\begingroup$

As the accepted answer states, the return at the current timestep is equal to the sum of discounted rewards from all future timesteps until the end of the episode. In Chapter 5 of Sutton and Barto, returns must be used to estimate the state-value and action-value functions because episode lengths are unrestricted and may be greater than one. In contrast, Chapter 2 deals with the very special case of multi-armed bandits in which episode lengths are always equal to one: The agent begins each episode in a fixed start state, takes an action, receives a reward, and then the episode terminates and the agent begins the next episode at the same start state. Therefore, a return is equivalent to a reward in Chapter 2 because all episodes have length one.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .