I am trying to solve the following exercise from Sutton and Barto:
Sutton and Barto Exercise 3.27 Give an equation for $\pi_*$ in terms of $q_*(s,a)$
However, I am struggling to do so. I know that $\pi_*$ is the policy which will pick the action with highest return, given we know what the optimal action values are. So intuitively, I would express the optimal policy like this: $$\pi_*(\text{argmax}_a q_*(s,a)|s) = 1$$. To express it like this: $$\pi_* = \text{argmax}_a q_*(s,a) $$
seems wrong since $\pi_*$ is a probability. What am I not getting correct here?