Suppose that in version 1 of a reinforcement-learning system an optimal policy $A$ got generated for executing a task. But, in a newer version 2 of that application (with new code changes), there might be some policy $B$ that would do slightly (1-2%) better than policy $A$.

How do you allow the system to learn that "better" policy $B$? I think the answer is retraining.

But during the training process, the old policy $A$ might still keep accumulating rewards delaying policy $B$ to be recognised as the "better" policy than $A$. This could get worse if each newer version of the system would contain a better policy which is only slightly better than the previous release's best policy. It would take a very long time to find the best policy.

Is this accepted in real-world RL systems? Or should I be figuring out a way to tell the system that "Hey, there might be a better policy somewhere, try to find that instead of rewarding existing policies."?

  • 1
    $\begingroup$ Yeah there are several approaches to detecting concept and model drift for ML. This article might help to get started: towardsdatascience.com/… $\endgroup$
    – user49143
    Aug 9, 2021 at 19:57


You must log in to answer this question.

Browse other questions tagged .