# MDP model for a unique system

Let's assume that we have a system where there are two exogenous factors, name $$a$$ and $$b$$, that can affect the system state. At each decision epoch either $$a$$ or $$b$$ occurs and it is not possible that they occur both at the same time. If $$a$$ occurs (with probability $$p(a)$$) the only action for the agent is to wait and reward is zero. If $$b$$ occurs (with probability $$p(b)$$) the agent has two options: action 1 and action 2 that happens immediately upon the occurrence of $$b$$. Actions 1 and 2 result in rewards $$r_1$$ and $$r_2$$ respectively. Since actions are done immediately upon the occurrence of $$b$$, I was wondering how I can define transition probabilities for this system.

• Can you confirm: (1) Are Actions 1, 2, & Wait the only actions possible for all states? (2) Is the state space finite? (3) For every state, action, and outcome, do you know the resulting state? – SecretAgentMan Nov 11 '20 at 2:23