# Questions tagged [markov-decision-process]

For questions related to Markov decision processes (MDPs), which model decision making in time-varying and usually stochastic environments.

13 questions
Filter by
Sorted by
Tagged with
47 views

### Understanding MDP's Dual Linear Program

I'm trying to understand a proof in Puterman'05 (Markov Decision Processes: Discrete Stochastic Systems). My question is within Theorem 6.9.1 pertaining to the equivalence of solutions to the primal (...
26 views

### MDP model for a unique system

Let's assume that we have a system where there are two exogenous factors, name $a$ and $b$, that can affect the system state. At each decision epoch either $a$ or $b$ occurs and it is not possible ...
87 views

### Markov Chain - Stationary or Borderline distribution [closed]

I have a Markov chain problem below, where : The problem An urn initially contains 3 black balls and 1 red ball. The balls are indistinguishable to the touch. One ball is randomly drawn. If this ball ...
35 views

### Equilibria in two player, finite horizon, zero sum stochastic game

Suppose we have a two player, finite horizon, zero sum stochastic game (a.k.a. "competitive MDP"). The game is defined over a finite set of state $S$. At each decision epoch $t\in\{1,\dots,T\}$ (for ...
82 views

### Queuing Theoretic Model with Memory

Consider a telephone company which receives call request at some arrival rate and serves each request with some service rate. This can be modeled using a Poisson Process. However I wish to model the ...
105 views

### Can I use an MDP for a stochastic inventory model when my demand distribution is non-stationary?

Most formulations of Markov Decision Processes for stochastic inventory models I've come across assume a fixed demand distribution. But in my case I have a time series forecast with a non stationary ...
1k views

### Number of aircraft to operate in an airline company

Suppose that an airline company has X planes, in general the companies keep a number of these aircraft (say 80% of X) in reserve in case of hazards. My question is how the airline companies compute ...
98 views

### Model or State Uncertainty in Queueing Model due to uncertain arrival rate

Introduction I am currently modelling a scenario where two queues need to be served by a single server in a non preemptive discipline. I am quite sorted on generating the optimal policy via Value or ...
199 views

### Algorithm for workforce scheduling for call volumes

I am trying to solve a workforce scheduling and optimization problem. Available data: daily level forecasted call volumes, shift schedules, resource utilization at the aggregate level, AHT at ...
1k views

### Infinite horizon versus finite horizon MDP

When can we approximate a finite horizon MDP with infinite horizon? Can we use infinite horizon stochastic shortest path problem on a directed acyclic graph?
167 views

### For the solution of complex gaming are we to leave behind approximation and fixed algorithms in favor of agents?

For an evolutionary extensive form game is the use of multi-agent systems a necessity or can more traditional approximate Nash equilibria and mean-field interactions suffice? Few games can be ...