Hot answers tagged

4

You are correct in the question that in RL terms chess a game of chess where the agent is one player, and the other player has an unknown state is a partially observable environment. Chess played like this is not a fully observable environment. I did not use the term "fully observable game" or "fully observable system" above , because ...


3

First, note that the current state does not determine the next state. What determines the next state are the dynamics of the environment, which, in the context of reinforcement learning and, in particular, MDPs, are encoded in the probability distribution $p(s', r \mid s, a)$. So, if the agent is in a certain state $s$, it could end up in another state $s'$, ...


2

Both Belief-MDPs and Bayes-Adaptive MDPs (BAMDPs) are special cases of POMDPs and their state space is augmented with a belief over their unobserved/hidden variables. In a belief-MDP, the hidden variables can change over the course of an episode. (Eg. Both the position and the uncertainty in the position of the robot can vary during an episode). In a BAMDP, ...


2

Your setting (of randomly dropping out reward signals) impacts expected future reward by multiply everything by a common factor $(1-\epsilon)$. As reinforcement learning (RL) control is based on maximising expected future reward, and multiplying by a positive constant does not affect ranking of action values, all existing RL methods will cope just fine ...


1

I guess it depends on what the goal is. If the goal is a general reward function, this formulation as an MPOMDP could make sense. One way to think about this, is as a way of modeling a general (centralized) POMDP with factored actions and observation spaces. However, it seems that what you are describing might be an active perception problem, where the goal ...


1

Yes, the core differences between the different categories of problems are correct as you've described them. For SMDPs, I'd like to remark that the water boiling example is maybe not the best. That looks more like an example of "delayed rewards", but not one of "durative actions": when the agent takes that action to raise the temperature, ...


Only top voted, non community-wiki answers of a minimum length are eligible