Questions tagged [pomdp]

For questions related to the concept of Partially Observable Markov Decision Process (POMDP), which is a generalization of the Markov Decision Process (MDP) to the cases where information about the states is incomplete (or partially observable).

Filter by
Sorted by
Tagged with
1
vote
0answers
16 views

Finding total number of states in a POMDP

I've been working on a question that is posed in a document I've been reading, that models qualifying for a job as a POMDP. In this model, a person takes 3 exams, and must pass all of them in order to ...
2
votes
0answers
28 views

Is there a way to do reinforcement learning in POMDP?

Are there any algorithms to use reinforcement learning to learn optimal policies in partially observable Markov decision process (POMDP) i.e. when the state is not perfectly observed. More ...
3
votes
2answers
508 views

Can Q-learning be used in a POMDP?

Can Q-learning (and SARSA) be directly used in a Partially Observable Markov Decision Process (POMDP)? If not, why not? My intuition is that the policies learned will be terrible because of partial ...
2
votes
1answer
40 views

Can the normalization factor for the belief state update be zero?

In order to update the belief state in a POMDP, the following formula is used: $$b'(s')=\frac{O(a, s', z) \sum_{s\in S} b(s)T(s, a, s')}{\mathbb{P}(z \mid b, a)}$$ where $s$ is a specific state in ...
2
votes
1answer
63 views

Does the observation function for POMDP always add up to 1?

I was reading in the article A tutorial on partially observable Markov decision processes (p. 120), by Michael L. Littman, that $\sum_{z \in Z}O(a, s',z) =1$, where $a$ is the action, $s'$ the next ...
5
votes
1answer
83 views

What is the appropriate approach to playing a game with incomplete state information?

I have a steady hex-map and turn-based war game featuring WWII carrier battles. I would like to improve the fixed policy for the AI using reinforcement learning. I have some beginner's questions, ...
1
vote
0answers
351 views

Is my understanding of the differences between MDP, Semi MDP and POMDP correct?

I just wanted to confirm that my understanding of the different Markov Decision Processes are correct, because they are the fundamentals of reinforcement learning. Also, I read a few literature ...
2
votes
2answers
87 views

How to define reward function in POMDPs?

How do I define a reward function for my POMDP model? In the literature, it is common to use one simple number as a reward, but I am not sure if this is really how you define a function. Because this ...