There are several occasion that reinforcement learning can be used as a control mean. The action is for example the set target temperature (which in many occasions change with time) and the state is for example the current temperature and other variables. The policy is then the control mean that is going to be learnt using the reinforcement learning.

As there is a dead time (input lag) and time delay in the real world, how can one propose to tackle this problem when using reinforcement learning as a control mean? Thank you.

  • $\begingroup$ Could you explain why you think a delay between action and observable effects of the action would be a problem? Generally in reinforcement learning it is not an issue, but maybe there is some special case for your example that I am missing. $\endgroup$ – Neil Slater Oct 30 '19 at 15:29
  • $\begingroup$ Related (maybe a duplicate?) ai.stackexchange.com/questions/8267/… $\endgroup$ – Neil Slater Oct 30 '19 at 15:31
  • $\begingroup$ I think you are right. This is the same problem that I ask here. I didn't find it before, but thank you. $\endgroup$ – JianNius Oct 30 '19 at 16:16
  • 1
    $\begingroup$ Possible duplicate of Dealing with Lags in Reinforcement Learning $\endgroup$ – JianNius Oct 30 '19 at 16:17
  • $\begingroup$ OK. If lag effects are extreme, such that observable state does not change at all, but "momentum" is building due to actions taken on each time step, and the target may overshoot, then you may need to make some changes to the state representation. If you think that applies in your case, then it may not be a duplicate. If so, add that detail to the question . . . otherwise it is good to have duplicates because it is another way to word the same question, so more chance of someone else looking to find the same answer. $\endgroup$ – Neil Slater Oct 30 '19 at 16:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.