Hot answers tagged

3

I will attempt to provide an answer to your questions based on the information you can find in the papers A Heuristic Variable Grid Solution Method for POMDPs (1997) by Ronen I. Brafman and Point-based value iteration: An anytime algorithm for POMDPs (2003) by Joelle Pineau et al. A grid-based approximate solution to a POMDP attempts to estimate a value ...


3

The usual (as presented in Reinforcement Learning: An Introduction) $Q$-learning and SARSA algorithms use (and update) a function of a state $s$ and action $a$, $Q(s, a)$. These algorithms assume that the current state $s$ is known. However, in POMDP, at each time step, the agent does not know the current state, but it maintains a "belief" (which, ...


2

I think that the normalisation factor is assumed to be non-zero. So, in practice, I guess, you must eventually check that $P(z \mid b, a)$ is non-zero (even though, I guess, it will likely never be zero because of round-off errors in computers). The formula to calculate $b'(s')$ comes from its definition, which is based on Bayes' theorem, where the ...


2

$O(a, s', z) = \mathbb{P}(z \mid a, s')$ is a conditional probability distribution, so it always needs to sum up to $1$. You should interpret $O(a, s', z)$ as the probability of observation $z$, given that the agent took action $a$ and landed in state $s'$. $O(a, s', z)$ is thus not a joint distribution, even though the notation $O(a, s', z)$ might suggest ...


1

The problem which you want to solve is Reinforcement Learning with Partially Observable Markov Decision Process. I recommend you taking a look at papers which work in this formalism. For example, papers related to dota bot created by OpenAI. You can start here: https://openai.com/five/


1

There is no major difference here between a POMDP and MDP. When setting reward values, you are generally trying to give the minimal information to the agent that when the sum of rewards is maximised, it solves the problem that you are posing. In literature it is common to use one simple number as a reward, but I am not sure if this is really how you ...


Only top voted, non community-wiki answers of a minimum length are eligible