Suppose there is a dynamic program that the state of the problem grows over time (more info is added to the state of the problem over time) and at each time, we need all historical data, full history, or all information gathered in a time window. My first question is if this model can be considered a Markov decision process? In an MDP, the state of the current time should be based on the state of the problem in the former time period. Here, this is true but we have gathered all information.
My second question is what are the general approaches to solve history-dependent dynamic programs? If we summarize information as a probability distribution, we would lose the exact information and our results will be suboptimal. I would be thankful if you can share some references for discrete and continuous-time problems.
Note: I asked this question in the following group, but I don't know if it is possible to transfer the question to this group or not. This link