Are there any algorithms to use reinforcement learning to learn optimal policies in partially observable Markov decision process (POMDP) i.e. when the state is not perfectly observed? More specifically, how does one update the belief state using Bayes' rule when the update Q kernel is not known?

  • $\begingroup$ What do you mean by "update Q kernel"? $\endgroup$
    – nbro
    Oct 8, 2019 at 0:29
  • $\begingroup$ So a POMDP has a state update kernel Q such that $x_{t+1} \sim Q(\cdot|x_t,a_t)$ and an observation kernel such that the observation $y_t\sim Q^o(\cdot|x_t)$. By update I mean the former, state update kernel $\endgroup$ Jan 10, 2021 at 22:26
  • $\begingroup$ Can you please link me to a research paper or book (or whatever) that uses the word "kernel"? Maybe this is only a terminology issue, but I don't think that "kernel" is the right word here. $\endgroup$
    – nbro
    Jan 10, 2021 at 22:31
  • 1
    $\begingroup$ Here is a paper link. In general you can google pomdp transition kernel to get many such results $\endgroup$ Jan 12, 2021 at 0:28
  • $\begingroup$ I am not sure if this is a duplicate of this one. I don't think so, as you're also asking a more specific question, but maybe you were just curious about how one could apply RL to POMDPs. Let me know. $\endgroup$
    – nbro
    Dec 19, 2021 at 18:53


You must log in to answer this question.

Browse other questions tagged .