Questions tagged [discount-factor]

Use for question involving discount factor (γ) in reinforcement learning.

Filter by
Sorted by
Tagged with
1
vote
1answer
43 views

When discounted MAB is useful?

Many of multi-armed bandit algorithms are used when the total reward is the sum of all rewards. However, in RL, the discounted reward is mainly used. Why the discounted reward is not prevailing in MAB ...
1
vote
2answers
53 views

How do I calculate the return given the discount factor and a sequence of rewards?

I know that $G_t = R_{t+1} + G_{t+1}$. Suppose $\gamma = 0.9$ and the reward sequence is $R_1 = 2$ followed by an infinite sequence of $7$s. What is the value of $G_0$? As it's infinite, how can we ...
2
votes
1answer
95 views

Updating action-value functions in Semi-Markov Decision Process and Reinforcement Learning

Suppose that the transition time between two states is a random variable (for example, unknown exponential distribution); and between two arrivals, there is no reward. If $\tau$ (real number not an ...
3
votes
2answers
145 views

What is the value of a state when there is a certain probability that agent will die after each step?

We assume infinite horizon and discount factor $\gamma = 1$. At each step, after the agent takes an action and gets its reward, there is a probability $\alpha = 0.2$, that agent will die. The assumed ...
3
votes
2answers
86 views

Are there any discount-factors based on branching factors?

I recently came across this function: $$\sum_{t = 0}^{\infty} \gamma^t R_t.$$ It's elegant and looks to be useful in the type of deterministic, perfect-information, finite models I'm working with. ...
3
votes
0answers
39 views

Does everyone still use discount rates?

In Section 10.4 of Sutton and Barto's RL book, they argue that the discount rate $\gamma$ has no effect in continuing settings. They show (at least for one objective function) that the average of the ...
2
votes
1answer
275 views

In online one step actor critic, why does the weights update become less significant as the episode progresses?

The Reinforcement Learning Book by Richard Sutton et al, section 13.5 shows an online actor critic algorithm. Why do the weights updates depend on the discount factor via $I$? It seems that the more ...
4
votes
1answer
411 views

Can the optimal value of discount factor in Deep Reinforcement Learning be between 0.2 to 0.8?

I'm now reading a book titled as Hands-On Reinforcement Learning with Python, and the author explains the discount factor that is used in Reinforcement Learing to discount the future reward, with the ...
5
votes
1answer
369 views

Is the discount not needed in a deterministic environment for Reinforcement Learning?

I'm now reading a book titled as "Deep Reinforcement Learning Hands-On" and the author said the following on the chapter about AlphaGo Zero: Self-play In AlphaGo Zero, the NN is used to ...