There are several occasion that reinforcement learning can be used as a control mean. The action is for example the set target temperature (which in many occasions change with time) and the state is for example the current temperature and other variables. The policy is then the control mean that is going to be learnt using the reinforcement learning.

As there is a dead time (input lag) and time delay in the real world, how can one propose to tackle this problem when using reinforcement learning as a control mean? Thank you.

  • $\begingroup$ Could you explain why you think a delay between action and observable effects of the action would be a problem? Generally in reinforcement learning it is not an issue, but maybe there is some special case for your example that I am missing. $\endgroup$ Commented Oct 30, 2019 at 15:29
  • $\begingroup$ Related (maybe a duplicate?) ai.stackexchange.com/questions/8267/… $\endgroup$ Commented Oct 30, 2019 at 15:31
  • $\begingroup$ I think you are right. This is the same problem that I ask here. I didn't find it before, but thank you. $\endgroup$
    – JianNius
    Commented Oct 30, 2019 at 16:16
  • 1
    $\begingroup$ Possible duplicate of Dealing with Lags in Reinforcement Learning $\endgroup$
    – JianNius
    Commented Oct 30, 2019 at 16:17
  • $\begingroup$ OK. If lag effects are extreme, such that observable state does not change at all, but "momentum" is building due to actions taken on each time step, and the target may overshoot, then you may need to make some changes to the state representation. If you think that applies in your case, then it may not be a duplicate. If so, add that detail to the question . . . otherwise it is good to have duplicates because it is another way to word the same question, so more chance of someone else looking to find the same answer. $\endgroup$ Commented Oct 30, 2019 at 16:41


You must log in to answer this question.