Recently, some work has been done planning and learning in Non-Markovian Decision Processes, that is, decision-making with temporally extended rewards. In these settings, a particular reward is received only when a particular temporal logic formula is satisfied (LTL or CTL formula). However, i cannot find any work about learning which rewards correspond to which temporally extended behavior.

In my searches, I came across k-order MDPs (which are non-Markovian). I did not find RL research done on k-order MDPs.

  • $\begingroup$ non-Markovian reward functions are quite old concepts. They were first introduced in the theory for control systems. You can take any objective fn. of a control system and apply planning and learning concepts to it. $\endgroup$ – abunickabhi Apr 7 '19 at 18:11
  • $\begingroup$ By "reward functions", do you mean "value functions"? $\endgroup$ – nbro Apr 8 '19 at 9:36
  • $\begingroup$ I don't mean the value function. I am assuming that the environment/society knows the rewards to give, and the agent wants to learn a model of what sequence of states visited results in what rewards. $\endgroup$ – Gavin Rens Apr 8 '19 at 19:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.