All Questions
Tagged with sutton-barto markov-decision-process
6 questions
4
votes
2
answers
261
views
$E_{\pi}[R_{t+1}|S_t=s,A_t=a] = E[R_{t+1}|S_t=s,A_t=a]$?
I would like to solve the first question of Exercise 3.19 from Sutton and Barto:
Exercise 3.19 The value of an action, $q_{\pi}(s, a)$, depends on the expected next reward and
the expected sum of the ...
- 1,105
0
votes
0
answers
27
views
What does the term "expected leaf node" in this exercise from Sutton-Barto mean?
What does the term "expected leaf node" in the Exercise below from Sutton-Barto mean?
- 1,105
4
votes
1
answer
126
views
Could you explain these 2 steps of the derivation of the Bellman equation as a recursive equation in Sutton & Barto?
I am reading the Sutton & Barto (2018) RL textbook.
On page 59, it derives the recursive property of the state-value function as below.
Could you explain the steps of third and fourth equality?
...
- 227
4
votes
0
answers
64
views
Does everyone still use discount rates?
In Section 10.4 of Sutton and Barto's RL book, they argue that the discount rate $\gamma$ has no effect in continuing settings. They show (at least for one objective function) that the average of the ...
- 41.4k
5
votes
2
answers
1k
views
Why does the definition of the reward function $r(s, a, s')$ involve the term $p(s' \mid s, a)$?
Sutton and Barto define the state–action–next-state reward function, $r(s, a, s')$, as follows (equation 3.6, p. 49)
$$
r(s, a, s^{\prime}) \doteq \mathbb{E}\left[R_{t} \mid S_{t-1}=s, A_{t-1}=a, S_{t}...
- 41.4k
3
votes
2
answers
887
views
Difference in continuing and episodic cases in Sutton and Barto - Introduction to RL, exercise 3.5
Excercise 3.5 The equastions in Section 3.1 are for the continuing
case and need to be modified (very slightly) to apply to episodic
tasks. Show that you know the modifications needed by giving ...
- 2,064