In Q-learning, I know that the Q-values are updated using the Bellman equation.
$$ Q^{new}(S_t,A_t) \leftarrow Q(S_t,A_t) + \alpha [R_{t+1} + \gamma \underset{a}{max} Q(S_{t+1},a) - Q(S_t,A_t)] $$
However, for the last state in the Q-table, the Q-values cannot be updated using the maximum Q-value of the next state term since the next state itself does not exist.
So how do we updated the Q-values for the last state? In my mind, the logical answer would be to continue using the Bellman equation but equalling the maximum Q-value for the next state term to 0, which results in:
$$ Q^{new}(S_t,A_t) \leftarrow Q(S_t,A_t) + \alpha [R_{t+1} - Q(S_t,A_t)] $$