How can I update the observation probability for a POMDP (or HMM), in order to have a more accurate prediction model?

The POMDP relies on observation probabilities that match an observation to a state. This poses an issue as the probabilities are not exactly known. However, the idea is to make them more accurate over time. The simplest idea would be to count the appeared observation as well as the states and use Naive Bayes estimators.

For example, $P(s' \mid a,s)$ is the probability that a subsequent state $s'$ is reached, given that the action $a$ and the previous state $s$ are known: In that simple case I can just count and then apply e.g. Naive Bayes estimators.

But, if I have an observation probability $P(z \mid s')$ (where $z$ is the observation) depending on a state, it's not as trivial to just count up the observation and the states, as I can not say that a state really was reached (Maybe I made an observation, but I was in a different state than wanted). I can just make an observation and hope I was in a certain state. But I can not say if e.g. I was in $s_1$ or maybe $s_2$. I think the update of the observation probability is only possible in the late aftermath.

So, what are good approaches to estimate my state?

  • 2
    $\begingroup$ Ok, now I think I fully understand this post. You want to estimate the probability distribution over the observations given e.g. the next states, $P(z \mid s')$, so that you can perform more accurate calculations (e.g. of the belief states) when that probability distribution is required. However, you don't know exactly in which state you are in, so you think that you cannot estimate $P(z \mid s')$, and that's why you're asking how you can estimate the state you're in. This looks like a circular problem. $\endgroup$
    – nbro
    Jan 8 at 0:05
  • 1
    $\begingroup$ Exactly, it is some sort of a circular problem, because the reliability of the observation is crucial to estimate the state. But if I can't even trust the observation (if I can't even trust my eyes) how will I determine that I was in a certain state. But I think, there must be a way to handle that problem, because how will ever a POMDP work, when I can't estimate my observation probability or if I can't have a more accurate propability distribution that matches the real systems behaviour. $\endgroup$
    – Pluxyy
    Jan 8 at 10:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.