Skip to main content

Questions tagged [expected-sarsa]

For questions related to the reinforcement learning algorithm called "expected SARSA" (as described in the book "Reinforcement Learning: An Introduction", by Sutton and Barto, 2nd edition).

Filter by
Sorted by
Tagged with
0 votes
0 answers

What are the update equations for Double Expected Sarsa with an $\epsilon$-greedy target policy?

This is question 6.13 in Sutton-Barto,page 136. What are the update equations for Double Expected Sarsa with an $\epsilon$-greedy target policy? The answer is given as follows: Let $Q_1$ and $Q_2$ ...
1 vote
0 answers

How to measure accuracy of learned value function of a fixed policy?

Let's say we've a given policy whose value function is to be evaluated. One way to get the value function can be using expected SARSA, as in this stack exchange answer. However, my MDP's state space ...
0 votes
1 answer

What does the figure in Q-learning vs Expected SARSA actually show?

I might be blind. But I wasn't able to find or figure out what the small difference between Q-learn and SARSA depicts in the following image; (src). What does the semi-circle show? and what does the ...
2 votes
1 answer

Why would SARSA diverge (but not Expected SARSA or Q-learning)?

In figure 6.3 (shown below) from Reinforcement Learning: An Introduction (second edition) by Sutton and Barto, SARSA is shown to perform worse asymptotically (after 100k episodes) than in the interim (...
6 votes
1 answer

Is Expected SARSA an off-policy or on-policy algorithm?

I understand that SARSA is an On-policy algorithm, and Q-learning an off-policy one. Sutton and Barto's textbook describes Expected Sarsa thusly: In these cliff walking results Expected Sarsa was ...
5 votes
1 answer

Expected SARSA vs SARSA in "RL: An Introduction" [closed]

Sutton and Barto state in the 2018-version of "Reinforcement Learning: An Introduction" in the context of Expected SARSA (p. 133) the following sentences: Expected SARSA is more complex ...