Can we say that $Q^\pi(s, a) = V^\pi(s)$
The correct relationship is this:
$$V^\pi(s) = \sum_a \pi(a|s) Q^\pi(s, a)$$
or, if you have a deterministic policy $a = \pi(s)$ you can instead write:
$$V^\pi(s) = Q^\pi(s, \pi(s))$$
Intuitively, this is because the $V^\pi(s)$ is the expected future return when following the policy $\pi$ from state $s$, whilst $Q^\pi(s, a)$ is the expected future return where it ignores the policy for only the next action $a$ (which will decide immediate reward $r$ and next state $s'$ independently of the policy), and thereafter follows $\pi$.
The above equations essentially show what happens when you apply the policy in state $s$ to decide which Q value(s) to use, they remove the independent choice of $a$ in $Q^\pi(s,a)$.
One possible misunderstanding that you have is that "on-policy" means the same thing as the equations show when considering action values (Q values) - it does not. When any algorithm learns action values, it learns the same thing conceptually, i.e. the expected (and maybe discounted) sum of future rewards given state $s$ when making a free choice of $a$ for the next step, and thereafter strictly following the policy being evaluated.
What is different between on-policy and off-policy is which policy gets evaluated - for on-policy methods like SARSA you evaluate action values for the same policy that you use to generate actions. For off-policy methods like Q learning you evaluate a different target policy. Both approaches have the same interpretation of what $Q(s,a)$ means otherwise, and have the same relationship between Q and V for their repsective policies.