An important property of a reinforcement learning problem is whether the environment of the agent is static, which means that nothing changes if the agent remains inactive. Different learning methods assume in varying degrees that the environment is static.

How can I check if and (if so) where in the Monte Carlo algorithm, temporal difference learning (TD(0)), the Dyna-Q architecture, and R-Max a static environment is implicitly assumed?

How could I modify the relevant learning methods so that they can in principle adapt to changing environments? (It can be assumed that $\epsilon$ is sufficiently large.)

  • $\begingroup$ Hi and welcome to this community! Can you please tell us where you saw this definition of a "static environment". There are the notions of a stochastic and deterministic environments, which do not seem to be what you're refering to. $\endgroup$ – nbro Jun 17 '19 at 19:39
  • $\begingroup$ It seems to me that the default assumption is static, because, even if learning based on real-time, real-world data, the parameters are static. Thus, while the environment might change, the model being utilized is static.. $\endgroup$ – DukeZhou Jun 24 '19 at 20:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.