# Why must the value of a state under an optimal policy equal the expected return for the best action from that state?

The Sutton and Barto reinforcement learning textbook states that

the value of a state under an optimal policy must equal the expected return for the best action from that state.

That is, $$v_*(s) = \max_a q_*(s, a).$$

I am having trouble gaining intuition for this. Since state values can be written as an expectation of the action values under a given policy, I am not sure I see how

$$v_*(s) = \sum_a \pi_*(a|s)q_*(s,a) = \max_a q_*(s, a).$$

I'd appreciate any insights!

You have an optimal policy $$\pi_*$$, and you are in the state $$s$$. Because the policy is optimal, it will only give probability to optimal actions. Let's say there are 5 actions $$a_1, ..., a_5$$ from your current state, and two of those are optimal, $$a_2$$ and $$a_4$$. Because they are both optimal, their action values will be equal $$q_*(s, a_2) = q_*(s, a_4) = q_\text{optimal}$$ and the optimal policy can decide to take either action in any possible ratio with the obvious restriction that the policy has to choose an action. This means that $$\pi_*(a_2|s) + \pi_*(a_4|s) = 1$$ because all the other actions are sub-optimal and the optimal policy would not take those actions, $$\pi_*(a_i|s) = 0$$ when $$i = 1, 3, 5$$. Then, you have:
\begin{align*} v_*(s) &= \sum_i \pi_*(a_i|s)q_*(s, a_i) \\ &= q_\text{optimal} \Big( \pi_*(a_2|s) + \pi_*(a_4|s) \Big) + \sum_{a \in a_1, a_3, a_5} \pi_*(a|s)q_*(s, a) \\ &= q_\text{optimal} \end{align*}
The optimal actions are optimal in this scenario because they have the largest action-value, so $$q_\text{optimal} = \max_a q_*(s, a)$$.
$$v_*(s)$$ is expected future returns given that you start from state $$s$$ and follow the optimal policy. $$q_*$$ is the expected future returns given that you start from state $$s$$ and take action $$a$$, then follow the optimal policy. So without the equations above, you can look at $$q_*(s, a)$$ as a one-step lookahead to evaluate all actions from the current state, and you take the action with the highest action value.