# Difference between Expected Return and Value Function in Reinforcement Learning

I've seen numerous mathematical explanations of reward, V(s) value functions, and returns functions. The reward provides an immediate return for being in a specific state. The better the reward, the better the state.

As I understand it, it can be better to be in a low-reward state sometimes because we can accumulate more long term which is where the expected returns function comes in. An expected return, return or cummulative reward function effectively adds up the rewards from the current state to the goal state. This implies it's model-based. However it seems a Value function does exactly the same?

Is a Value function a Return function? Or are they different?

• Why did you revert by edit? I tried to improve the clarity of your question. Now, given that it is hard to understand it, I downvote it (because I don't want to lose 10 minutes again to decipher what you are talking about). – nbro Feb 15 '19 at 11:28
• @nbro sorry but I don't care about votes. You changed it to the point where was too generic. The terms I've used are found in common books. I know you want the points but I'd rather the question be helpful to people struggling as I was. Students use this site who aren't as clear as yourself on whether something referred to in a book is EXACTLY the same as another - the adjectives help a great deal. Those who understand will figure out what I'm talking about. Those who don't may get here BECAUSE of the keywords and understand the answer below. – user3168961 Feb 15 '19 at 13:04
• I've never seen terms like "expected returns function", "cummulative reward function", or sentences like "we can accumulate more long term" (which is quite nonsensical). I've already read some literature and I don't recall these terms, which I think you either read them from some web article (which you shouldn't be reading in the first place, if that's the case) or you made them up. Can you please tell us exactly where you found these terms?? – nbro Feb 15 '19 at 13:06
• Yes, I tried to make your question as general and thus useful as possible. As it is now, it's very specific to your case, so it will not be useful to a lot of people. – nbro Feb 15 '19 at 13:09
• @nbro I found refs to "Expected Reward" in Reinforcement Learning by Richard S. Sutton. Otherwise, probably the university assignment I had. I searched at the time and I found the descriptions vague. This answer helped me. Otherwise I don't know. Why would someone make up words? I wasn't lonely dude, I was just struggling with RL :-) – user3168961 Feb 15 '19 at 13:27