First of all, $Q_\pi(s, a)$ IS DEFINED AS the value (i.e. the expected return) of taking some action $a$ in some state $s$, AND THEN following some given policy $\pi$ (until e.g. the end of the game or your life). In other words, suppose that you take action $a$ in state $s$, AND THEN use the policy $\pi$ to behave in the world until you die, then $Q_\pi(s, a)$ would represent the value that you would obtain.
So, we are DEFINING $Q_\pi(s, a)$ in a certain way. This is a DEFINITION! It's not an algorithm. In the algorithms (e.g. Q-learning), things will typically change, but that's a different story that you should investigate later.
From this, I infer that the $Q$ value (the action-value function) will be affected by the policy $\pi$.
So, $Q_\pi(s, a)$ will not keep changing. You could say that $Q_\pi(s, a)$ (which is a function) is "affected by" $\pi$ ONLY in the sense that it is "defined in terms of" $\pi$. To be precise, $Q_\pi(s, a)$ is actually an expectation (which is a mathematical concept similar to an ideal average). If you are not familiar with the concept of expectation, I suggest you get familiar with it first, before studying reinforcement learning.
Shouldn't the Q value be constant, because the same action taken in the same state will always give the same yield (and hence remain constantly good/bad)?
Again, there's the distinction between the algorithm that you actually use to find the function $Q_\pi(s, a)$ and the definition of the same function. In case you are estimating the function with an algorithm, you will not necessarily find "constant Q values". It depends on different aspects, which I would like to avoid discussing here, so that this post doesn't become an open discussion (I suggest you first learn about the basic Bellman equations and then you study the algorithms from the book Reinforcement learning: an introduction by Sutton and Barto).
