# Questions tagged [discount-factor]

Use for question involving discount factor (γ) in reinforcement learning.

9 questions
Filter by
Sorted by
Tagged with
53 views

### How do I calculate the return given the discount factor and a sequence of rewards?

I know that $G_t = R_{t+1} + G_{t+1}$. Suppose $\gamma = 0.9$ and the reward sequence is $R_1 = 2$ followed by an infinite sequence of $7$s. What is the value of $G_0$? As it's infinite, how can we ...
95 views

### Updating action-value functions in Semi-Markov Decision Process and Reinforcement Learning

Suppose that the transition time between two states is a random variable (for example, unknown exponential distribution); and between two arrivals, there is no reward. If $\tau$ (real number not an ...
145 views

### What is the value of a state when there is a certain probability that agent will die after each step?

We assume infinite horizon and discount factor $\gamma = 1$. At each step, after the agent takes an action and gets its reward, there is a probability $\alpha = 0.2$, that agent will die. The assumed ...
43 views

### When discounted MAB is useful?

Many of multi-armed bandit algorithms are used when the total reward is the sum of all rewards. However, in RL, the discounted reward is mainly used. Why the discounted reward is not prevailing in MAB ...
39 views

### Does everyone still use discount rates?

In Section 10.4 of Sutton and Barto's RL book, they argue that the discount rate $\gamma$ has no effect in continuing settings. They show (at least for one objective function) that the average of the ...
275 views

### In online one step actor critic, why does the weights update become less significant as the episode progresses?

The Reinforcement Learning Book by Richard Sutton et al, section 13.5 shows an online actor critic algorithm. Why do the weights updates depend on the discount factor via $I$? It seems that the more ...
86 views

### Are there any discount-factors based on branching factors?

I recently came across this function: $$\sum_{t = 0}^{\infty} \gamma^t R_t.$$ It's elegant and looks to be useful in the type of deterministic, perfect-information, finite models I'm working with. ...