# How to model history-dependent dynamic program?

Suppose there is a dynamic program that the state of the problem grows over time (more info is added to the state of the problem over time) and at each time, we need all historical data, full history, or all information gathered in a time window. My first question is if this model can be considered a Markov decision process? In an MDP, the state of the current time should be based on the state of the problem in the former time period. Here, this is true but we have gathered all information.

My second question is what are the general approaches to solve history-dependent dynamic programs? If we summarize information as a probability distribution, we would lose the exact information and our results will be suboptimal. I would be thankful if you can share some references for discrete and continuous-time problems.

Note: I asked this question in the following group, but I don't know if it is possible to transfer the question to this group or not. This link

• Concerning 2) You state that results will be suboptimal if summarizing (i.e. extracting) information. Does that mean that you are looking for an exact method? Commented Jun 19, 2022 at 20:12
• Not an exact solution, but I am willing to solve it with an error bound. I am wondering if I store all historical data as a probability distribution, is the problem transformed to a POMDP? For example, instead of storing all customers' age, I use a probability distribution that matches the data.
– Amin
Commented Jun 20, 2022 at 2:44
• Even though Markov can be considered a loose statement it usually indicates "memorylessness" (no info beyond the current state). Also if you are thinking of dynamic programming in LP or MIP context you will be handling decision variables at all stages. Therefore you will be using all history with something that can be called "markovian" (perhaps). Can you provide a minimal application example rather than a general statement? Commented Jun 20, 2022 at 8:43
• Suppose we have perishable products in the inventory and also some lead times for orders. In this case, we need to store the lifetime of products and also former orders.
– Amin
Commented Jun 25, 2022 at 18:46
• Can't you describe that without the full information of those orders? So lets say you have one product: Then you can have in your state the amount of goods that perishes in 1 day, the amount that perishes in 2 days (lets say up to n days) etc. And for ordered goods the same: The amount that will arrive in 1 day, 2 days ..... Of course that will be quite large so you might use e.g. approximate dynamic programming (ADP) to solve it. Commented Jun 27, 2022 at 21:09