# Are the two policies in SARSA for choosing an action the same?

Here is the pseudocode for SARSA (which I took from here)

Are the two policies in SARSA for choosing an action equal? I guess yes, because it is called an on-policy learning algorithm. But could I, for example, also use different policies in this SARSA framework? For example an $$\epsilon$$-greedy policy and a softmax policy. Maybe the resulting algorithm would not be called SARSA anymore but it would be something similar.

• I have deleted my answer as it caused too many follow-up questions, thus did not seem to be helping you. I highly recommend reading the book Reinforcement Learning: An Introduction, which covers the rationale behind basic value-based methods very well: incompleteideas.net/book/RLbook2020.pdf Dec 16 '21 at 18:08
• @NeilSlater I think you should undelete your answer. It answers the question. If the OP is not able to understand it, it's a different matter.
– nbro
Dec 16 '21 at 23:36
• @nbro: OK, I have undeleted the question. Dec 17 '21 at 7:44

However, the usual practice is to have just one active policy for SARSA, typically $$\epsilon$$-greedy based on Q values learned so far. It is worth noting here, that as Q values change during learning - which happens on each time step - then the policy may also change (when the greedy action choice changes). So even when you use the same rules to derive the policy in SARSA, the actual policy used may vary, even in the middle of the loop. In that respect, the SARSA algorithm uses many policies, but typically only one approach to determining the current policy.