As your reference mentioned below:

When the state-value function is used to assess actions in this way it is called a critic, and the overall policy-gradient method is termed an actor–critic method. Note that the bias in the gradient estimate is not due to bootstrapping as such; the actor would be biased even if the critic was learned by a Monte Carlo method.

Thus this inherent bias in actor-critic algo faces the same convergence issue as any TD boostrapping learning algo whose convergence is a theoretical result from stochastic approximation methods as discussed here.

converges with probability 1 to an optimal policy and action-value function, under the usual conditions on the step sizes (2.7), as long as all state–action pairs are visited an infinite number of times and the policy converges in the limit to the greedy policy... but not for the case of constant step-size parameter, $α_n(a)=α$. In the latter case, the second condition is not met, indicating that the estimates never completely converge but continue to vary in response to the most recently received rewards. As we mentioned above, this is actually desirable in a nonstationary environment, and problems that are effectively nonstationary are the most common in reinforcement learning.

But even this is the case it's is often desirable for the same reason that bootstrapping TD methods are often superior to Monte Carlo methods due to substantially reduced variance.

Sutton & Barto's book of Reinforcement Learning mentions below:

When the state-value function is used to assess actions in this way it is called a critic, and the overall policy-gradient method is termed an actor–critic method. Note that the bias in the gradient estimate is not due to bootstrapping as such; the actor would be biased even if the critic was learned by a Monte Carlo method... The natural state-value-function learning method to pair with this is semi-gradient TD(0).

Thus this inherent bias in actor-critic algo using one-trajectory sample during every episode to update both its actor's and critic's parameters at every step (see page 332 for its algo detail) faces the same convergence issue as any TD boostrapping learning algo whose convergence is a theoretical result from stochastic approximation methods as discussed here and the relevant source in the same reference is summarized below. So long as the advantage function as approximated by the difference of the one-step return and the current state value as the baseline (both values are from the same critic) converges to their true values just like TD state values learning, the policy actor and critic parameters would optimize the policy's performance metric per the policy gradient theorem.

converges with probability 1 to an optimal policy and action-value function, under the usual conditions on the step sizes (2.7), as long as all state–action pairs are visited an infinite number of times and the policy converges in the limit to the greedy policy... but not for the case of constant step-size parameter, $α_n(a)=α$. In the latter case, the second condition is not met, indicating that the estimates never completely converge but continue to vary in response to the most recently received rewards. As we mentioned above, this is actually desirable in a nonstationary environment, and problems that are effectively nonstationary are the most common in reinforcement learning.

Therefore actor-critic algo usually needs tons of data for training in order to approximately satisfy above convergence conditions, yet it's often superior mainly due to substantially reduced variance.