I'm now reading a book titled as Hands-On Reinforcement Learning with Python, and the author explains the discount factor that is used in Reinforcement Learing to discount the future reward, with the following:
A discount factor of 0 will never learn considering only the immediate rewards; similarly, a discount factor of 1 will learn forever looking for the future reward, which may lead to infinity. So the optimal value of the discount factor lies between 0.2 to 0.8.
The author seems to be not going to explain further about the figure, but all the tutorials and explanations I have ever read write the optimal (or at least widely used) discount factor between 0.9 an 0.99. This is the first time I have seen such a low-figure discount factor.
All the other explanations the author makes regarding the discount factor are the same as I have read so far.
Is the author correct here or does it depend on cases? If it is, then what kind of problems and/or situations should I set the discount factor as low as such figure at?
I just found the following answer at Quora:
Of course. A discount factor of 0 will never learn, meanwhile a factor near of 1 will only consider the last learning. A factor equal or greater than 1 will cause the not convergence of the algorithm. Values usually used are [0.2, 0.8]
EDIT: That was the learning factor. The discount factor only affect how you use the reward. For a better explanation:
State-Action-Reward-State-Action - Wikipedia
See influences of variables .
I don't know what is written in the question as it in not visible in Quora, but it seems that the 0.2 to 0.8 figure is used for learning factor, not discount factor. Maybe the author is confused with it...? I'm not sure what the learning factor is, though.