Questions tagged [sarsa]

For questions related to the reinforcement learning (on-policy) algorithm called SARSA, which stands for (s, a, r, s', a').

9 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
3 votes
0 answers
98 views

Evaluation a policy learned using Q - learning

I have been reading literature on reinforcement learning in healthcare. I am slightly confused between the policy evaluation for both SARSA and Q-learning. To my knowledge, I believe that SARSA is ...
calveeen's user avatar
  • 1,261
2 votes
0 answers
184 views

Reduction of state space of the game Connect Four to apply RL algorithms SARSA and Q-Learning

I would like to implement the reinforcement learning algorithms SARSA and Q-Learning for the board game Connect Four. I am familiar with the algorithms and know about their limitations regarding large ...
legnib's user avatar
  • 21
2 votes
0 answers
85 views

Can $Q$-learning or SARSA be thought of a Markov Chain?

I might just be overthinking a very simple question but nonetheless the following has been bugging me a lot. Given an MDP with non-trivial state and action sets, we can implement the SARSA algorithm ...
dezdichado's user avatar
2 votes
0 answers
117 views

Is Deep SARSA learning a feasible approach?

I noticed that SARSA has been rarely used in the deep RL setting. Usually, the training for DQN is done off-policy. I think one of the major reasons for this is due to greater sample efficiency in ...
calveeen's user avatar
  • 1,261
2 votes
0 answers
177 views

Tic-tac-toe: How would standard SARSA and Q-learning yield different results in the agent's behaviour?

I know this is deceptively simple. Tic tac toe is a well studied game for RL. Assume your agent is playing aggainst a strong opponent. I know you deal in after states. I know that in Q learning the ...
JimSi's user avatar
  • 121
2 votes
1 answer
452 views

Why isn't my implementation of DQN using TensorFlow on the FrozenWorld environment working?

I am trying to test DQN on FrozenWorld environment in gym using TensorFlow 2.x. The update rule is (off policy) $$Q(s,a) \leftarrow Q(s,a)+\alpha (r+\gamma~ max_{a'}Q(s',a')-Q(s,a))$$ I am using an ...
kosa's user avatar
  • 121
1 vote
0 answers
85 views

Why does importance sampling ratio start and end one step later in off-policy SARSA given in Sutton-Barto's RL book?

In Sutton & Barto's RL book (page 149) they say: Sarsa update can be completely replaced by a simple off-policy form $Q_{t+n}(S_t,A_t)=Q_{t+n−1}(S_t,A_t) + \rho_{t+1:t+n} [G_{t:t+n} − Q_{t+n−1}(...
DSPinfinity's user avatar
0 votes
0 answers
45 views

What else could I try to avoid catastrophic forgetting in my implementation of Semi-Gradient SARSA?

I was trying to implement the semi-gradient SARSA algorithm (see p.244 Reinforcement Learning: An Introduction) using PyTorch: ...
kklaw's user avatar
  • 175
0 votes
0 answers
81 views

Reinforcement learning for rearranging the mobile home screen icon layout: what inputs/states do I need to pass into the algorithm?

I have a problem where I need to rearrange a particular user's mobile home screen icon layout. Let's say that the social media app usage of a user is high compared to other app usage. So I need the ...
George  Silva's user avatar