1
$\begingroup$

Sutton-Barto, Chapter 6, page:130

By 8000 time steps, the greedy policy was long since optimal (a trajectory from it is shown inset); continued $\epsilon$-greedy exploration kept the average episode length at about 17 steps, two more than the minimum of 15. enter image description here

What is meant here by the first sentence: "By 8000 time steps, the greedy policy was long since optimal (a trajectory from it is shown inset)"?

I am confused also: the applied policy is $\epsilon$-greedy SARSA but in the sentence there is "greedy policy". Is the path shown from S to G for greedy policy, or for SARSA?

$\endgroup$

1 Answer 1

2
$\begingroup$

Underpinning an epsilon-greedy policy is a deterministic greedy policy. What they are saying here is that this underlying policy has been near optimal for quite a few episodes.

They are still talking about this deterministic greedy policy when discussing the inset.

$\endgroup$
2
  • $\begingroup$ I am confused by your explanation as well. is the one given in figure (from S to G) greedy or epsilon-greedy? $\endgroup$ Apr 25 at 0:17
  • $\begingroup$ My interpretation is greedy as it is the optimal path in this scenario. Theoretically it could be a trajectory from epsilon greedy under which the epsilon is sufficiently low that all moves were made deterministically or that an exploration move was taken but it happened to be the same as the greedy move. The simpler explanation is that it is simply greedy. $\endgroup$
    – foreverska
    Apr 25 at 0:25

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .