New answers tagged sutton-barto
2
votes
Accepted
$E_{\pi}[R_{t+1}|S_t=s,A_t=a] = E[R_{t+1}|S_t=s,A_t=a]$?
Question: Can I write it without the subscript? So $$E_{\pi}[R_{t+1}|S_t=s,A_t=a] = E[R_{t+1}|S_t=s,A_t=a]$$
Yes, your reasoning is sound, there is no need to condition the expectation on the policy, ...
- 23.8k
4
votes
Accepted
In Value Iteration, why can we initialize the value function arbitrarily?
If the value function of a state $v(s)$ is relatively high, then you are absolutely correct in saying that a greedy policy may choose to visit $s$, since the high $v(s)$ makes it very promising. The ...
- 928
6
votes
In Value Iteration, why can we initialize the value function arbitrarily?
Is this something to do with the Bellman optimality constraint itself?
That is part of it, and important for episodic problems without discounting. The Bellman equations link between time steps, ...
- 23.8k
1
vote
Accepted
Why does the average-reward estimator for continuing tasks use the TD error?
Mystery solved thanks to Exercise 10.8 in the book. The reason is that we want the running mean to converge to the actual value of the average reward.
With $\bar{R}_{t + 1} = \bar{R}_t + \beta \delta$...
- 66
Top 50 recent answers are included