# Epsilon-greedy action selection question

Is the following sentence correct?

The estimated values $$Q(a)$$ do not converge to the true values $$q_*(a)$$ because $$\epsilon$$-greedy action selection behaves randomly from time to time.

My Answer: The sentence is wrong since eventually the estimated value $$Q(a)$$ will converge to $$q_*(a)$$ and the randomness of epsilon greedy action selection won't be able to influence it if we run it for a long time. Is my answer correct?

• @Faizy Thanks but could you please explain why? I don't get why randomness is a problem for the convergence of estimated Q to the true value q. Commented Oct 18, 2022 at 15:23
• i think you correct.... Running the epsilon greedy algorithm for a long time will minimize the influence of randomness on the estimated value $Q(a)$. The epsilon greedy action randomly selects actions with probability epsilon and otherwise acts greedily with respect to the current estimated value of $Q(a)$. Over time, the algorithm will tend to converge on the true values of $q_∗(a)$, since the random actions will be less influential. Commented Oct 18, 2022 at 18:22