Questions tagged [sarsa]
For questions related to the reinforcement learning (on-policy) algorithm called SARSA, which stands for (s, a, r, s', a').
9
questions
10
votes
1
answer
6k
views
Can Q-learning be used in a POMDP?
Can Q-learning (and SARSA) be directly used in a Partially Observable Markov Decision Process (POMDP)? If not, why not? My intuition is that the policies learned will be terrible because of partial ...
- 298
1
vote
1
answer
333
views
What is meant by "two action selections" in SARSA?
I have some difficulties understanding the difference between Q-learning and SARSA. Here (What are the differences between SARSA and Q-learning?) the following updating formulas are given:
Q-Learning
$...
- 256
8
votes
2
answers
2k
views
How should I handle action selection in the terminal state when implementing SARSA?
I recently started learning about reinforcement learning. Currently, I am trying to implement the SARSA algorithm. However, I do not know how to deal with $Q(s', a')$, when $s'$ is the terminal state. ...
- 562
3
votes
1
answer
8k
views
What is the difference between the $\epsilon$-greedy and softmax policies?
Could someone explain to me which is the key difference between the $\epsilon$-greedy policy and the softmax policy? In particular, in the contest of SARSA and Q-Learning algorithms. I understood the ...
- 199
2
votes
1
answer
782
views
What are the differences between SARSA and Q-learning? [closed]
From Sutton and Barto's book Reinforcement Learning (Adaptive Computation and Machine Learning series), are the following definitions:
To aid my learning of RL and gain an intuition, I'm focusing on ...
- 335
1
vote
2
answers
2k
views
What are the conditions for the convergence of SARSA to the optimal value function?
Is it correct that for SARSA to converge to the optimal value function (and policy)
The learning rate parameter $\alpha$ must satisfy the conditions:
$$\sum \alpha_{n^k(s,a)} =\infty \quad \text{and}...
- 309
11
votes
2
answers
2k
views
Are Q-learning and SARSA the same when action selection is greedy?
I'm currently studying reinforcement learning and I'm having difficulties with question 6.12 in Sutton and Barto's book.
Suppose action selection is greedy. Is Q-learning then exactly the same ...
- 131
3
votes
1
answer
170
views
Convergence of epsilon greedy policy (with no epsilon decay) using TD Learning?
If I create a policy using the q-values of an epsilon greedy policy using the Sarsa algorithm (not changing the epsilon with each episode), will it converge to the optimal solution to the MDP? I am ...
3
votes
1
answer
2k
views
How to deal with the terminal state in SARSA in a multi-agent setting?
I'm training a SARSA agent to update a Q function, but I'm confused about how you handle the final state. In this case, when the game ends and there is no $S'$.
For example, the agent performed an ...
- 403