# Does the observation function for POMDP always add up to 1?

I was reading in the article A tutorial on partially observable Markov decision processes (p. 120), by Michael L. Littman, that $$\sum_{z \in Z}O(a, s',z) =1$$, where $$a$$ is the action, $$s'$$ the next possible state and $$z$$ a certain/specific observation.

How come that the observation function $$O(a, s', z)$$ adds up to $$1$$ in POMDP?

## 1 Answer

$$O(a, s', z) = \mathbb{P}(z \mid a, s')$$ is a conditional probability distribution, so it always needs to sum up to $$1$$. You should interpret $$O(a, s', z)$$ as the probability of observation $$z$$, given that the agent took action $$a$$ and landed in state $$s'$$.

$$O(a, s', z)$$ is thus not a joint distribution, even though the notation $$O(a, s', z)$$ might suggest it. In this case, $$O(a, s', z)$$ simply means that $$O$$ is a function of $$a$$, $$z$$ and $$s'$$.

If you want to see a proof that conditional probability distributions sum up to 1, have a look at this post.

• But does it still mean, that I have to observe a state somehow, even the information might not be accurate? – Bryan McGill Mar 28 '19 at 17:22
• @BryanMcGill You don't observe any state (in POMDPs). You get "observations", which you use to model your belief in which state you might be in. – nbro Mar 28 '19 at 17:25
• That clarifies everythinga. I came along some illustrations where "observations" were connected to states and I got confused. Maybe you could edit your answer regarding that one sentence in the beginning. – Bryan McGill Mar 28 '19 at 17:32