The Sutton and Barto reinforcement learning textbook states that ``the value of a state under an optimal policy must equal the expected return for the best action from that state''. That is,
$$v_*(s) = \max_a q_*(s, a).$$
I am having trouble gaining intuition for this. Since state values can be written as an expectation of the action values under a given policy, I am not sure I see how
$$v_*(s) = \sum_a \pi_*(a|s)q_*(s,a) = \max_a q_*(s, a).$$
I'd appreciate any insights! Thanks so much!