Last call to make your voice heard! Our 2022 Developer Survey closes in less than a week. Take survey.

# Tag Info

### Are Q-learning and SARSA the same when action selection is greedy?

If we write the pseudo-code for the SARSA algorithm we first initialise our hyper-parameters etc. and then initialise $S_t$, which we use to choose $A_t$ from our policy $\pi(a|s)$. Then for each $t$ ...
• 4,035
Accepted

### Is Q-learning only capable of learning a deterministic policy?

If we assume a tabular setting, then Q-learning converges to the optimal state-action value function, from which an optimal policy can be derived, provided a few conditions are met. In finite MDPs, ...
• 33.2k
Accepted

### When do SARSA and Q-Learning converge to optimal Q values?

The true answers are 1 and 3. 1 is true because the required conditions for tabular Q-learning to converge is that each state action pair will be visited infinitely often, and Q-learning learns ...
• 4,035
Accepted

### How should I handle action selection in the terminal state when implementing SARSA?

The value $Q(s', ~\cdot~)$ should always be implemented to simply be equal to $0$ for any terminal state $s'$ (the dot instead of an action as second argument there indicates that what I just wrote ...
• 9,326
Accepted

### Can Q-learning be used in a POMDP?

The usual (as presented in Reinforcement Learning: An Introduction) $Q$-learning and SARSA algorithms use (and update) a function of a state $s$ and action $a$, $Q(s, a)$. These algorithms assume that ...
• 33.2k
Accepted

### Expected SARSA vs SARSA in "RL: An Introduction"

Why is the action selection random with Sarsa? A policy could be stochastic. In the case of SARSA, it is stochastic because of the use of $\epsilon$-greedy. Isn't it on-policy and therefore ϵ-...
• 490

• 23.3k
1 vote
Accepted

### What are the differences between SARSA and Q-learning?

The main difference between the two is that Q-learning is an off policy algorithm. That is, we learn about an policy that is different to the one we choose to make actions. To see this, lets look at ...
• 4,035

Only top scored, non community-wiki answers of a minimum length are eligible