New answers tagged markov-decision-process
2
votes
Accepted
Policy performance when the stationary state distribution is not unique in RL
Indeed your idealized problem is an edge case which is less studied. Most practical problem-solving RL literature assumes that induced MCs from MDPs are ergodic which simplifies both theory and ...
- 5,387
2
votes
Can the limiting distribution depend on the initial distribution?
Indeed the limiting behavior of a non-ergodic Markov chain which is not irreducible with multiple mean positive recurrent communicating classes depends on its initial distribution, since apparently ...
- 5,387
1
vote
Accepted
Can the limiting distribution depend on the initial distribution?
all definitions I find say that the limiting distribution does not depend on the initial distribution
The proof of this trait of the limit relies on the asumption of ergodicity.
You can define an MDP ...
- 33.3k
2
votes
Accepted
Ergodic MDP: does it have to be aperiodic?
The apparent issue here is that you are interpreting every visited state in a trajectory as part of the recurrent class which is incorrect. Only the limit behavior of the policy matters, which is ...
- 5,387
2
votes
Why are ergodic MDPs also communicating?
There are inconsistencies in the definition of an ergodic Markov Chain while de concepts of irreducibility, aperiodicity and recurrence are well grounded.
Thus, I think it is better not to employ this ...
1
vote
Accepted
Conditions for the existence of stationary state distribution
Besides stationary policy whose decision at each state doesn't depend on time in a MDP context, the ergodic criterion for the policy-induced Markov chain (MC) is irreducibility of the said chain and ...
- 5,387
2
votes
Accepted
Why are ergodic MDPs also communicating?
If all states in an irreducible Markov chain (MC) are ergodic, then the chain is said to be ergodic. So once an ergodic MC is induced by a policy, then by above definition all states are irreducible. ...
- 5,387
Top 50 recent answers are included