Tag Info

Short answer Two policies are different if they take different actions in a specific state $s$ (or they give different probabilities of taking those actions in $s$). There can be more than one optimal policy for a given value function: this only happens when two actions have the same value in a given state. Nevertheless, both policies lead to the same ...

Yes, in general any linear combination of probability distributions between optimal policies is also an optimal policy. In fact any combination with each state treated separately will also be an optimal policy. This can be seen using the equation for optimal deterministic policy in terms of optimal value function: $$\pi^*(s) = \text{argmax}_a [\sum_{r,s'}p(r,...