1
$\begingroup$

If I have a DQN, and I care A LOT about future rewards (moreso than current rewards), can I set gamma to a number greater than 1? Like 1.1 perhaps?

$\endgroup$

2 Answers 2

2
$\begingroup$

$ \gamma $ goes up to 1, but cannot be greater than or equal to 1 (this would make the discounted reward infinite).

The discount factor $ \gamma $ determines the importance of future rewards. A factor of 0 will make the agent "myopic" (or short-sighted) by only considering current rewards, while a factor approaching 1 will make it strive for a long-term high reward. If the discount factor meets or exceeds 1, the action values may diverge. For $ \gamma =1$, without a terminal state, or if the agent never reaches one, all environment histories become infinitely long, and utilities with additive, undiscounted rewards generally become infinite.

Source:https://en.wikipedia.org/wiki/Q-learning https://cs.stanford.edu/people/karpathy/reinforcejs/puckworld.html

$\endgroup$
1
  • 1
    $\begingroup$ +1. Exactly, simply set gamma to 1, or something very close to it. Even the value of 0.99 generally works fine, and still avoids the terminal state search issue. $\endgroup$ Apr 8, 2019 at 19:36
1
$\begingroup$

You can't! As if you have a $\gamma$ greater than $1$ the specified sum for the q-learning will diverge! (goes to infinity in the future steps for $\gamma^n$). To know more about that please more scrutinize on the specified formula for the q-learning.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .