New answers tagged markov-decision-process
2
Aside from the points raised in nbro's answer, I'd like to point out that for a single MDP (a single instance of a "problem"), it may be sensible to study it from perspectives that include no policy at all, or multiple different policies.
For instance, if I have an MDP, I may be interested in studying it by looking at various inherent properties of the ...
6
The MDP defines the environment (which corresponds to the task that you need to solve), so it defines e.g. the states of the environment, the actions that you can take in those states, the probabilities of transitioning from one state to the other and the probabilities of getting a reward when you take a certain action in a certain state.
The policy ...
2
Isn't the environment constantly changing in this game?
The current state of the agent and the environment is constantly changing as you play, but not necessarily the transition probabilities. For simplicity, you may assume that the transition probabilities do not change (e.g. if the dealer and the deck are the same every time you play).
How would the ...
0
To my knowledge you can't compute or solve an uncountably large MDP numerically. It will need to be discretized in some capacity. The same applies for classic control: you can't optimize over the true functional so you use a discrete approximation to the system and solve that.
1
Note: I assume you mean, countable Action and State Sets by 'Finite'.
MDP(s) are not exclusive to finite spaces only. They can be used in Continuous/uncountable sets of Action and States too.
Markov Decision Process (MDP) is a tuple $(\mathcal S, \mathcal A, \mathcal P^a_s, \mathcal R^a_{ss'}, \gamma, \mathcal S_o)$ where $\mathcal S$ is a set of States, $\...
1
In addition to the reason outlined in the comment, also note that if the state-space and action-space are both finite and of feasible size, tabular methods can be used, and there are some advantages to them (like the existence of convergence guarantees and generally a smaller number of hyperparameters to tune).
0
Transition Probabilities: Consider that you are at state $s$ and from that state take an action $a$.Then there are some probability you will land up at state $s_{1}'$ or $s_{2}'$ ($s'$ indicate the next states). Those probabilities are called transition probabilities. In this example, the transition matrix is just a 3D array since it depends on your state ...
Top 50 recent answers are included
