# How to check if the state of a dynamic program is Markovian or not?

Based on Markov chain context, we say a stochastic process is Markovian if the state at time $$t+1$$, $$S_{t+1}$$ just depends on the state in the previous step, that is, $$\Pr\left( S_{t+1}|S_1, S_2, \cdots, S_t \right)= \Pr\left( S_{t+1}|S_t \right)$$.

Now consider a dynamic program where we need to store all historical data up to time $$t$$ ($$\mathcal{H}_t$$) as our State variable, that is, $$S_t=\mathcal{H_t}$$. Now, suppose that at time $$t+1$$ a new data like $$(t+1, a_{t+1})$$ is observed and the historical data is updated as $$\mathcal{H}_{t+1}=\mathcal{H}_t \cup \{(t+1, a_{t+1}) \}$$ and so $$S_{t+1}= \mathcal{H}_{t+1}$$.

In this case, we could write the state of the problem based on the state of the previous period but we still need to store all historical data. I am wondering if this problem can be considered as a Markovian model?

In the former case, suppose we extend our problem to a continuous-time case. How is it possible to write the state transition formula? Actually, new data are added based on a new Poisson process. So, the time between two new pieces of data follows an exponential distribution.