I'm training a SARSA agent to update a Q function, but I'm confused about how you handle the final state. In this case, when the game ends and there is no $S'$.
For example, the agent performed an action based on the state $S$, and, because of that, the agent won or lost and there is no $S'$ to transition to.
So, how do you update the Q function with the very last reward in that scenario, given that the state hasn't actually changed? That case $S'$ would equal $S$ even though an action was performed and the agent received a reward (they ultimately won or lost, so quite important update to make!).
Do I add extra inputs to the state "agent won" and "game finished", and that's the difference between $S$ and $S'$ for the final Q update?
To make clear, this is in reference to a multi-agent/player system. So, the final action the agent takes could have a cost/reward associated with it, but the subsequent actions other agents then take could further determine a greater gain or loss for this agent and whether it wins or loses. So, the final state and chosen action, in effect, could generate different rewards without the agent taking further actions.