Tag Info

Accepted

Is the discount not needed in a deterministic environment for Reinforcement Learning?

The motivation for adding the discount factor $\gamma$ is generally, at least initially, based simply in "theoretical convenience". Ideally, we'd like to define the "objective" of an RL agent as ...
• 10.4k
Accepted

If $\gamma \in (0,1)$, what is the on-policy state distribution for episodic tasks?

This question is really getting at the meaning of the discount factor in Markov decision processes. There are actually two, equivalent ways of interpreting the discount factor. The first is probably ...
• 1,281
Accepted

For episodic tasks with an absorbing state, why can't we both have $\gamma=1$ and $T= \infty$ in the definition of the return?

$T = \infty$ and $\gamma = 1$ cannot be both true at the same time because the return defined in equation 3.11 is supposed to be a unified definition of the return for both continuing and episodic ...
• 40.9k
Accepted

What should the discount factor for the non-slippery version of the FrozenLake environment be?

After trying to understand what was happening by going through the algorithm on paper (below), I found all values were the same, so greedifying with respect to the value function often just had the ...
• 32.7k

What is the value of a state when there is a certain probability that agent will die after each step?

The value of a state depends on the policy that you use, so I'll make the assumption here that you're talking about value using the optimal policy. According to the optimal policy, the agent would ...
Accepted

Should the concept of discounted rewards result in multiple arrays per episode in RL?

The discount factor is not used to take into account uncertainty, but to encourage the agent to have a longer term view when it takes an action. If the discount factor is close to 0, the agent is ...

Can the optimal value of discount factor in Deep Reinforcement Learning be between 0.2 to 0.8?

The discount factor is not something you should be optimising. It is typically part of the problem statement. For practical purposes, you may set it below 1.0 for continuous problems when in fact you ...
• 32.7k

• 32.7k
1 vote
Accepted

Are there any discount-factors based on branching factors?

First, an important note on any form of discounting: adding a discount factor can change what the optimal policy is. The optimal policy when a discount factor is present can be different from the ...
• 10.4k

Only top scored, non community-wiki answers of a minimum length are eligible