3
$\begingroup$

I had the same question when I am reading the RL textbook from Sutton Bartol as posted here.

enter image description here

Why do we update $W$ with $\frac{1}{\mu (A_t | S_t)}$ instead of $\frac{\pi (A_t | S_t)}{\mu (A_t | S_t)}$?

It seems that, with the updating rule from the textbook, whatever action $\mu$ decides to choose, we automatically assume that $\pi$ will choose it with 100% probability. But $\pi$ is greedy with respect to Q. How does this assumption make sense?

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ It says there that $\pi$ is a deterministic policy $\endgroup$
    – Brale
    Commented May 5, 2020 at 17:43
  • 1
    $\begingroup$ Ah I see! It could be 0 but in case of 0, loop already breaks before updating W. Should have read the algorithms a few more times. Thank you very much! $\endgroup$
    – roy
    Commented May 5, 2020 at 18:15

0

Reset to default

You must log in to answer this question.

Browse other questions tagged .