Let's assume that we have a system where there are two exogenous factors, name $a$ and $b$, that can affect the system state. At each decision epoch either $a$ or $b$ occurs and it is not possible that they occur both at the same time. If $a$ occurs (with probability $p(a)$) the only action for the agent is to wait and reward is zero. If $b$ occurs (with probability $p(b)$) the agent has two options: action 1 and action 2 that happens immediately upon the occurrence of $b$. Actions 1 and 2 result in rewards $r_1$ and $r_2$ respectively. Since actions are done immediately upon the occurrence of $b$, I was wondering how I can define transition probabilities for this system.

  • $\begingroup$ Can you confirm: (1) Are Actions 1, 2, & Wait the only actions possible for all states? (2) Is the state space finite? (3) For every state, action, and outcome, do you know the resulting state? $\endgroup$ – SecretAgentMan Nov 11 '20 at 2:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.