2022 Developer Survey is open! Take survey.

# Questions tagged [semi-mdp]

For questions related to semi-Markov Decision Processes (SMDP).

8 questions
Filter by
Sorted by
Tagged with
1 vote
37 views

### How to model a multi-agent reinforcement learning problem where actions of different agents can take different durations?

I am confused on a conceptual scale how I would be able to model a multi-agent reinforcement learning problem when each agent performing an action would take different durations to complete the action....
• 203
1 vote
164 views

### Bellman optimality equation in semi Markov decision process

I wrote a Python program for a simple inventory control problem where decision epochs are equally divided (every morning) and there is no lead time for orders (the time between submitting an order ...
• 13
138 views

### Updating action-value functions in Semi-Markov Decision Process and Reinforcement Learning

Suppose that the transition time between two states is a random variable (for example, unknown exponential distribution); and between two arrivals, there is no reward. If $\tau$ (real number not an ...
• 451
36 views

### Relationship between the reward rate and the sampled reward in a Semi-Markov Decision Process

In the paper: Reinforcement learning methods for continuous-time Markov decision problems, the authors provide the following update rule for the Q-learning algorithm, when applied to Semi-Markov ...
• 682
3k views

### What are options in reinforcement learning?

According to a lecture (week 10) about Reinforcement Learning [1], the concept of an option allows searching the state space of an agent much faster. The lecture was hard to follow because many new ...
75 views

### How to apply or extend the $Q(\lambda)$ algorithm to semi-MDPs?

I want to model an SMDP such that time is discretized and the transition time between the two states follows an exponential distribution and there would be no reward between the transition. Can I ...
• 451
136 views

### Should I model my problem as a semi-MDP?

I have a system (like a bank) that people (customers) are entered into the systems by a Poisson process, so the time between the arrival of people (two consecutive customers) will be a random variable....
• 451