# Behaviour policy must be stochastic in states where it is not identical to the Optimal policy

In Sutton & Barto Reinforcement learning book, page 103 (chapter: off-policy learning via importance sampling), the following statement is given:

"In order to use episodes from $$b$$ to estimate values for $$\pi$$, we require that every action taken under $$\pi$$ is also taken, at least occasionally, under $$b$$. That is, we require that $$\pi$$(a|s) > 0 implies $$b$$(a|s) > 0. This is called the assumption of coverage. It follows from coverage that $$b$$ must be stochastic in states where it is not identical to $$\pi$$. "

where $$b$$ refers to a behavior policy and $$\pi$$ to the target optimal policy. I don't understand how the above conclusion was obtained from the definition of coverage.

Say we are in state $$s$$, and $$b$$ chooses $$a'$$ but $$\pi$$ chooses $$a$$. Since $$\pi$$ chose $$a$$ ($$\pi(a | s) >0$$), coverage implies that $$b(a|s)>0$$. But $$a'$$ was ultimately chosen by $$b$$, so it must be that $$b(a'|s)>0$$. Hence there is some chance that $$a'$$ is chosen and there is some chance that $$a$$ is chosen, meaning $$b$$ is stochastic in state $$s$$. Thus the above conclusion follows from the definition of coverage when the policies are not identical under $$s$$.