Bear in mind $\nabla_{\theta} J$ is used to update parameters $\theta$ in each step and $r(\tau)$ is the total reward of a sampling trajectory $\tau$ where we can rewrite the policy gradient equation as $$\nabla_{\theta} J(\theta) \approx r(\tau) \sum_{t\ge 0} \nabla_{\theta} \log \pi_{\theta}\left(a_{t} \mid s_{t}\right)$$ If $r(\tau)$ is positively great then the above policy gradient ascent algorithm intuitively intends to update the parameters in a direction so that each term $\log \pi_{\theta}\left(a_{t} \mid s_{t}\right)$ is pushed up. And since the $\log$ function is monotone, each term $\pi_{\theta}\left(a_{t} \mid s_{t}\right)$ is pushed up too which is nothing but exploitation viewed from the thesis of exploration vs exploitation of RL. This is why your reference claims it might seem simplistic to say that if a trajectory is good then all its actions were good.
If this cannot convince you, let's use a simple grid world example where an agent starts at the left corner and aims to reach a goal at the right corner. Suppose there are only two possible actions in any state $s$: $UP$ and $RIGHT$, and initial probabilities of taking these actions under the policy $\pi_{\theta}$ are $\pi_{\theta}(UP∣s)=0.5,\pi_{\theta}(RIGHT∣s)=0.5$. The reward $r(τ)$ for a trajectory $τ$ is $+10$ if it reaches the goal using the action $RIGHT$. For simplicity, let's say the softmax parameterization for actins in any state $s$ is: $π_θ(UP∣s)=\frac{e^{θ_1}}{e^{θ_1}+e^{θ_2}}, π_θ(RIGHT∣s)=\frac{e^{θ_2}}{e^{θ_1}+e^{θ_2}}$. Apparently, only by choosing action $RIGHT$ at each state $s$ the parameters $\theta$ will be pushed up in the direction of $θ_2$ per above policy gradient algorithm and thus $π_θ(RIGHT∣s)=\frac{e^{θ_2}}{e^{θ_1}+e^{θ_2}}$ is pushed up as well.
