0
$\begingroup$

Here is the pseudocode for SARSA (which I took from here)

enter image description here

Why are we choosing more than 1 action in SARSA? One for going into the next state and the other one for updating the Q function?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

Why are we choosing more than 1 action in SARSA?

There is never a state where more than one action is chosen.

The appearance of two Choose statements is an artifact of the loop design and variable management in the pseudocode.

One for going into the next state and the other one for updating the Q function?

Sort of. There is only ever one chosen action for each state.

However, you need to have two actions in scope - the current action (just taken and observed) and planned next action, in order to process the update rule. Hence there are two variables a and a' and the code needs to generate one or other outside the main loop, or have some other way to ensure that it has access to current and next values. This is also why at the end of the loop, the "next" values $s', a'$ get copied to the "current" values $s, a$.

Improve this answer
$\endgroup$
0

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .