Artificial Intelligence Stack Exchange is a question and answer site for people interested in conceptual questions about life and challenges in a world where "cognitive" functions can be mimicked in purely digital environment. It only takes a minute to sign up.
Sign up to join this community
Here is the pseudocode for SARSA (which I took from here)
Do we only select one action at the very beginning and then we always choose the same action for each step? Does it really make sense to choose the same initially chosen action $a$ regardless of the state $s$?
Do we only select one action at the very beginning and then we always choose the same action for each step?
No.
The pseudocode is clear on this, by using the word Choose and referencing a policy.
If you were expected to take the same action again, then the pseudocode already has the previous action in variable a, so it would not need to state anything about making a choice or using a policy.
The $a \leftarrow a'$ notation is common way to describe copying values*, so the variable a is changed at the end of each loop.
Does it really make sense to choose the same initially chosen action $a$ regardless of the state $s$?
Not in this case. Some learning algorithms do use a form of "sticky" exploration where a single exploratory action is committed to for multiple time steps. It can be useful in some environments. But not basic SARSA as described in the question.
* It avoids the ambiguity of $=$ as assignment or equality operator. You may also see $:=$ for assignment as an alternative, but for instance Sutton & Barto use $\leftarrow$ consistently, and a lot of RL literature follows this convention.
