# Questions tagged [discount-factor]

Use for question involving discount factor (γ) in reinforcement learning.

9 questions
Filter by
Sorted by
Tagged with
369 views

### Is the discount not needed in a deterministic environment for Reinforcement Learning?

I'm now reading a book titled as "Deep Reinforcement Learning Hands-On" and the author said the following on the chapter about AlphaGo Zero: Self-play In AlphaGo Zero, the NN is used to ...
411 views

### Can the optimal value of discount factor in Deep Reinforcement Learning be between 0.2 to 0.8?

I'm now reading a book titled as Hands-On Reinforcement Learning with Python, and the author explains the discount factor that is used in Reinforcement Learing to discount the future reward, with the ...
145 views

### What is the value of a state when there is a certain probability that agent will die after each step?

We assume infinite horizon and discount factor $\gamma = 1$. At each step, after the agent takes an action and gets its reward, there is a probability $\alpha = 0.2$, that agent will die. The assumed ...
86 views

### Are there any discount-factors based on branching factors?

I recently came across this function: $$\sum_{t = 0}^{\infty} \gamma^t R_t.$$ It's elegant and looks to be useful in the type of deterministic, perfect-information, finite models I'm working with. ...
39 views

### Does everyone still use discount rates?

In Section 10.4 of Sutton and Barto's RL book, they argue that the discount rate $\gamma$ has no effect in continuing settings. They show (at least for one objective function) that the average of the ...
95 views

### Updating action-value functions in Semi-Markov Decision Process and Reinforcement Learning

Suppose that the transition time between two states is a random variable (for example, unknown exponential distribution); and between two arrivals, there is no reward. If $\tau$ (real number not an ...
275 views

### In online one step actor critic, why does the weights update become less significant as the episode progresses?

The Reinforcement Learning Book by Richard Sutton et al, section 13.5 shows an online actor critic algorithm. Why do the weights updates depend on the discount factor via $I$? It seems that the more ...
I know that $G_t = R_{t+1} + G_{t+1}$. Suppose $\gamma = 0.9$ and the reward sequence is $R_1 = 2$ followed by an infinite sequence of $7$s. What is the value of $G_0$? As it's infinite, how can we ...