# Tag Info

Accepted

### Is the discount not needed in a deterministic environment for Reinforcement Learning?

The motivation for adding the discount factor $\gamma$ is generally, at least initially, based simply in "theoretical convenience". Ideally, we'd like to define the "objective" of an RL agent as ...
• 9,419
Accepted

### If $\gamma \in (0,1)$, what is the on-policy state distribution for episodic tasks?

This question is really getting at the meaning of the discount factor in Markov decision processes. There are actually two, equivalent ways of interpreting the discount factor. The first is probably ...
• 931
Accepted

### For episodic tasks with an absorbing state, why can't we both have $\gamma=1$ and $T= \infty$ in the definition of the return?

$T = \infty$ and $\gamma = 1$ cannot be both true at the same time because the return defined in equation 3.11 is supposed to be a unified definition of the return for both continuing and episodic ...
• 34.3k
Accepted

### What should the discount factor for the non-slippery version of the FrozenLake environment be?

After trying to understand what was happening by going through the algorithm on paper (below), I found all values were the same, so greedifying with respect to the value function often just had the ...
• 24.5k

### What is the value of a state when there is a certain probability that agent will die after each step?

The value of a state depends on the policy that you use, so I'll make the assumption here that you're talking about value using the optimal policy. According to the optimal policy, the agent would ...

### Can the optimal value of discount factor in Deep Reinforcement Learning be between 0.2 to 0.8?

The discount factor is not something you should be optimising. It is typically part of the problem statement. For practical purposes, you may set it below 1.0 for continuous problems when in fact you ...
• 24.5k

• 24.5k
1 vote
Accepted

### Are there any discount-factors based on branching factors?

First, an important note on any form of discounting: adding a discount factor can change what the optimal policy is. The optimal policy when a discount factor is present can be different from the ...
• 9,419

Only top scored, non community-wiki answers of a minimum length are eligible