8
$\begingroup$

Most formulations of Markov Decision Processes for stochastic inventory models I've come across assume a fixed demand distribution.

But in my case I have a time series forecast with a non stationary distribution and seasonal behavior. Is it still possible to formulate the problem as an MDP?

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ If you have non-stationary demand but use a finite-horizon MDP, then it might make sense. I struggle to envision how an infinite-horizon MDP would make sense since there is no "steady-state." $\endgroup$ – SecretAgentMan Nov 5 '19 at 13:43
2
$\begingroup$

You can still use the MDP modeling approach for your system with unknown or non-stationary request distributions.

I think the paper: MDP formulation and solution algorithms for inventory management with multiple suppliers and supply and demand uncertainty, is a good start point to deepen your search. Also, look at POMDPs(Partially Observable MDPs).

In the literature, there are also some applications of learning techniques and MDPs in parallel to manage the uncertainty in the system. For example:

  • Imani, Mahdi, Seyede Fatemeh Ghoreishi, and Ulisses M. Braga-Neto. "Bayesian control of large MDPs with unknown dynamics in data-poor environments." Advances in neural information processing systems. 2018.
Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.