Could someone explain the red line (especially, the meaning of the difference) in prioritized sweeping algorithm below?

Sutton-Barto, page 170:

enter image description here

  • $\begingroup$ Why are you so persistent in using screen shots of text, with no readable summary? You know better by now. Please stop doing this. $\endgroup$ Commented May 25 at 7:16
  • $\begingroup$ I will try your follow your advice from now on unless the writing will take too much time. $\endgroup$ Commented May 25 at 16:54
  • 2
    $\begingroup$ Thank you. You don't have to replicate an entire screenshot, but you should always reproduce the relevant parts to the question in text (i.e. don't underline text in an image), and ideally enough of the rest to give a little context. Here, copying line (e) as text instead of underlining it in an image would probably be enough. $\endgroup$ Commented May 26 at 6:58

1 Answer 1


$ R + \gamma {max}_aQ(S^{\prime},a) $ is the target Q value for $(S,A)$. The present estimate $Q(S,A)$ is subtracted from it to arrive at the error. The absolute value is taken to arrive at the absolute error. This absolute error is used to prioritize the next state to update.


Not the answer you're looking for? Browse other questions tagged .