# How do we express $q_\pi(s,a)$ as a function of $p(s',r|s,a)$ and $v_\pi(s)$?

The task (exercise 3.13 in the RL book by Sutton and Barto) is to express $$q_\pi(s,a)$$ as a function of $$p(s',r|s,a)$$ and $$v_\pi(s)$$.

$$q_\pi(s,a)$$ is the action-value function, that states how good it is to be at some state $$s$$ in the Markov Decision Process (MDP), if at that state, we choose an action $$a$$, and after that action, the policy $$\pi(s,a)$$ determines future actions.

Say that we are at some state $$s$$, and we choose an action $$a$$. The probability of landing at some other state $$s'$$ is determined by $$p(s',r|s,a)$$. Each new state $$s'$$ then has a state-value function that determines how good is it to be at $$s'$$ if all future actions are given by the policy $$\pi(s',a)$$, therefore:

$$q_\pi(s,a) = \sum_{s' \in S} p(s',r|s,a) v_\pi(s')$$

Is this correct?

Not quite. You are missing the reward at time step $$t+1$$.
The definition you are looking for is (leaving out the $$\pi$$ subscripts for ease of notation)
$$q(s,a) = \mathbb{E}[R_{t+1} + \gamma v(s') | S_t=s,A_t=a] = \sum_{r,s'}(r + v(s'))p(s',r|s,a)\;.$$
Because $$q(s,a)$$ relates to expected returns at time $$t$$, and returns are defined as $$G_t = \sum_{b = 0}^\infty \gamma ^b R_{t+b+1}$$, thus $$R_{t+1}$$ is also a random variable at time $$t$$ that we need to take expectation with respect to, not just the state that we transition into.