Can I use an MDP for a stochastic inventory model when my demand distribution is non-stationary?

Question

Most formulations of Markov Decision Processes for stochastic inventory models I've come across assume a fixed demand distribution.

But in my case I have a time series forecast with a non stationary distribution and seasonal behavior. Is it still possible to formulate the problem as an MDP?

Answer 1

You can still use the MDP modeling approach for your system with unknown or non-stationary request distributions.

I think the paper: MDP formulation and solution algorithms for inventory management with multiple suppliers and supply and demand uncertainty, is a good start point to deepen your search. Also, look at POMDPs(Partially Observable MDPs).

In the literature, there are also some applications of learning techniques and MDPs in parallel to manage the uncertainty in the system. For example:

• Imani, Mahdi, Seyede Fatemeh Ghoreishi, and Ulisses M. Braga-Neto. "Bayesian control of large MDPs with unknown dynamics in data-poor environments." Advances in neural information processing systems. 2018.