# Questions tagged [sarsa]

For questions related to the reinforcement learning (on-policy) algorithm called SARSA, which stands for (s, a, r, s', a').

44 questions
Filter by
Sorted by
Tagged with
94 views

### Convergence of epsilon greedy policy (with no epsilon decay) using TD Learning?

If I create a policy using the q-values of an epsilon greedy policy using the Sarsa algorithm (not changing the epsilon with each episode), will it converge to the optimal solution to the MDP? I am ...
58 views

### Understanding SARSA with binary features and function approximation

I'm reading the Sutton and Barto Book on Reinforcement Learning (for oral exam preparation). However, I'm stuck on the algorithm for SARSA($\lambda$) with binary features and linear function ...
153 views

### Help on Deep Sarsa algorithm that work with pytorch (Adam optimiser) but not with keras/Tensorflow (Adam optimiser)

I have a deep sarsa algorithm wich work great on Pytorch on lunar-lander-v2 and I would use with Keras/Tensorflow. It use mini-batch of size 64 wich are used 128 time to train at each episode. There ...
312 views

### How to perform the back propagation step in Semi-gradient SARSA using a deep neural network?

For the back weight update step, I need to calculate $\nabla\hat{q}(S,A,w)$. My neural network takes in the state vector $S$ and gives out the action values for state $S$ and each action in the action ...
734 views

### Is Q-learning only capable of learning a deterministic policy?

I was following a reinforcement learning course on coursera and in this video at 2:57 the instructor says Expected SARSA and SARSA both allow us to learn an optimal $\epsilon$-soft policy, but, Q-...
70 views

### Can we also estimate $V_{\pi}$ with SARSA?

For SARSA, I know we can estimate the action value $Q(s,a)$, and the relationship between $V(s)$ and $Q(s,a)$ is $V_{\pi}(s) = \sum_{a \in \mathcal{A}} \pi(a|s)Q_{\pi} (s,a)$. So my question is, can ...
242 views

### Is it possible learning convergence is lost in Reinforcement Learning as the state space grows?

I am new in the AI field and I am trying to use Reinforcement Learning. Specifically, I am using tabular Q-Learning and SARSA algorithms to solve a sequential decision making problem. (I am using <...
1 vote
73 views

5k views

### Can Q-learning be used in a POMDP?

Can Q-learning (and SARSA) be directly used in a Partially Observable Markov Decision Process (POMDP)? If not, why not? My intuition is that the policies learned will be terrible because of partial ...
4k views

### Expected SARSA vs SARSA in "RL: An Introduction" [closed]

Sutton and Barto state in the 2018-version of "Reinforcement Learning: An Introduction" in the context of Expected SARSA (p. 133) the following sentences: Expected SARSA is more complex ...
2k views

### How to deal with the terminal state in SARSA in a multi-agent setting?

I'm training a SARSA agent to update a Q function, but I'm confused about how you handle the final state. In this case, when the game ends and there is no $S'$. For example, the agent performed an ...
I recently started learning about reinforcement learning. Currently, I am trying to implement the SARSA algorithm. However, I do not know how to deal with $Q(s', a')$, when $s'$ is the terminal state. ...