Questions tagged [markov-decision-process]

For questions related to Markov decision processes (MDPs), which model decision making in time-varying and usually stochastic environments.

Filter by
Sorted by
Tagged with
3
votes
1answer
76 views

Queuing Theoretic Model with Memory

Consider a telephone company which receives call request at some arrival rate and serves each request with some service rate. This can be modeled using a Poisson Process. However I wish to model the ...
7
votes
1answer
54 views

Can I use an MDP for a stochastic inventory model when my demand distribution is non-stationary?

Most formulations of Markov Decision Processes for stochastic inventory models I've come across assume a fixed demand distribution. But in my case I have a time series forecast with a non stationary ...
12
votes
3answers
1k views

Number of aircraft to operate in an airline company

Suppose that an airline company has X planes, in general the companies keep a number of these aircraft (say 80% of X) in reserve in case of hazards. My question is how the airline companies compute ...
11
votes
1answer
76 views

Model or State Uncertainty in Queueing Model due to uncertain arrival rate

Introduction I am currently modelling a scenario where two queues need to be served by a single server in a non preemptive discipline. I am quite sorted on generating the optimal policy via Value or ...
10
votes
1answer
138 views

Algorithm for workforce scheduling for call volumes

I am trying to solve a workforce scheduling and optimization problem. Available data: daily level forecasted call volumes, shift schedules, resource utilization at the aggregate level, AHT at ...
10
votes
2answers
204 views

Infinite horizon versus finite horizon MDP

When can we approximate a finite horizon MDP with infinite horizon? Can we use infinite horizon stochastic shortest path problem on a directed acyclic graph?
9
votes
0answers
129 views

For the solution of complex gaming are we to leave behind approximation and fixed algorithms in favor of agents?

For an evolutionary extensive form game is the use of multi-agent systems a necessity or can more traditional approximate Nash equilibria and mean-field interactions suffice? Few games can be ...
11
votes
1answer
100 views

Open source Markov decision process solvers

Which sources are out there that provide reliable and fast MDP solvers? I prefer that the library is callable from C++ but other languages are fine too. There is indeed an abundance of repositories ...
8
votes
1answer
77 views

Convergence rate value iteration

Value iteration is one of the most commonly used methods to solve Markov decision processes. Its convergence rate obviously depends on the number of states and actions. However, the convergence rate ...