Questions tagged [transition-model]

For questions related to the transition model of a Markov decision process. This term is often used in reinforcement learning (RL) to distinguish between model-based and model-free RL algorithms, where model-based algorithms use the transition model while model-free don't use it.

Filter by
Sorted by
Tagged with
3
votes
1answer
126 views

How can we find the value function by solving a system of linear equations without knowing the policy?

An MDP is a Markov Reward Process with decisions, it’s an environment in which all states are Markov. This is what we want to solve. An MDP is a tuple $(S, A, P, R, \gamma)$, where $S$ is our state ...
4
votes
1answer
63 views

How should I implement the state transition when it is a Gaussian distribution?

I am reading this paper Anxiety, Avoidance and Sequential Evaluation and is confused about the implementation of a specific lab study. Namely, the authors model what is called the Balloon task using a ...
8
votes
1answer
164 views

How to fill in missing transitions when sampling an MDP transition table?

I have a simulator modelling a relatively complex scenario. I extract ~12 discrete features from the simulator state which forms the basis for my MDP state space. Suppose I am estimating the ...
6
votes
1answer
2k views

What are the state space and the state transition function in AI?

I'm studying for my AI final exam, and I'm stuck in the state space representation. I understand initial and goal states, but what I don't understand is the state space and state transition function. ...
0
votes
2answers
78 views

Does “transition model” alone in an MDP imply it's non-deterministic?

I am looking at a lecture on POMDP, and the context is that, when the quadcopter can't see the landmarks, it has to use reckoning. And then he mentions the transition model is not deterministic, hence ...