2
$\begingroup$

I was reading in the article A tutorial on partially observable Markov decision processes (p. 120), by Michael L. Littman, that $\sum_{z \in Z}O(a, s',z) =1$, where $a$ is the action, $s'$ the next possible state and $z$ a certain/specific observation.

How come that the observation function $O(a, s', z)$ adds up to $1$ in POMDP?

Improve this question
$\endgroup$
0

1 Answer 1

Reset to default
2
$\begingroup$

$O(a, s', z) = \mathbb{P}(z \mid a, s')$ is a conditional probability distribution, so it always needs to sum up to $1$. You should interpret $O(a, s', z)$ as the probability of observation $z$, given that the agent took action $a$ and landed in state $s'$.

$O(a, s', z)$ is thus not a joint distribution, even though the notation $O(a, s', z)$ might suggest it. In this case, $O(a, s', z)$ simply means that $O$ is a function of $a$, $z$ and $s'$.

If you want to see a proof that conditional probability distributions sum up to 1, have a look at this post.

Improve this answer
$\endgroup$
3
  • $\begingroup$ But does it still mean, that I have to observe a state somehow, even the information might not be accurate? $\endgroup$
    – Bryan McGill
    Commented Mar 28, 2019 at 17:22
  • 1
    $\begingroup$ @BryanMcGill You don't observe any state (in POMDPs). You get "observations", which you use to model your belief in which state you might be in. $\endgroup$
    – nbro
    Commented Mar 28, 2019 at 17:25
  • $\begingroup$ That clarifies everythinga. I came along some illustrations where "observations" were connected to states and I got confused. Maybe you could edit your answer regarding that one sentence in the beginning. $\endgroup$
    – Bryan McGill
    Commented Mar 28, 2019 at 17:32

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .