Questions tagged [markov-property]

For questions related to the Markov property or Markov assumption (that is, the assumption that the "future is independent of the past, given the present"), which underlies e.g. most reinforcement learning algorithms.

Filter by
Sorted by
Tagged with
1
vote
0answers
13 views

What are the most common non-Markov RL paradigms?

I am interested in doing model-free RL but not using the Markov assumptions typical for MDPs or POMDPs. What are alternative paradigms that don't rely on the Markov assumptions? Are there any common ...
4
votes
1answer
40 views

How is the Markovian property consistent in reinforcement learning based scheduling?

In Reinforcement Learning, an MDP model incorporates the Markovian property. A lot of scheduling applications in a lot of disciplines use reinforcement learning (mostly deep RL) to learn scheduling ...
3
votes
1answer
522 views

What does the Markov assumption say about the history of state sequences?

Does the Markov assumption say that the conditional probability of the next state only depends on the current state or does it say that the conditional probability depends on a fixed finite number of ...
4
votes
1answer
80 views

How to assign rewards in a non-Markovian environment?

I am quite new to the Reinforcement Learning domain and I am curious about something. It seems to be the case that the majority of current research assumes Markovian environments, that is, future ...
2
votes
2answers
60 views

Can non-Markov environments also be deterministic?

The definition of deterministic environment I am familiar with goes as follows: The next state of the agent depends only on the current state and the action chosen by the agent. By exclusion, ...
2
votes
1answer
48 views

Is the Markov property assumed in the forward algorithm?

I'm majoring in pure linguistics (not computational), and I don't have any basic knowledge regarding computational science or mathematics. But I happen to take "Automatic Speech Recognition" course in ...
4
votes
0answers
38 views

What research has been done on learning non-Markovian reward functions?

Recently, some work has been done planning and learning in Non-Markovian Decision Processes, that is, decision-making with temporally extended rewards. In these settings, a particular reward is ...
1
vote
0answers
45 views

What limitations does the Markov property place on real time learning?

The Markov property is the dependence of a system's future state probability distribution solely on the present state, excluding any dependence on past system history. The presence of the Markov ...