I have a Markov chain problem below, where :

The problem

An urn initially contains 3 black balls and 1 red ball. The balls are indistinguishable to the touch. One ball is randomly drawn.

If this ball is black, it is removed. If this ball is red, we put it back in the urn. The operation is repeated until the urn contains only the red ball.

1- Let Xn be the number of black balls contained in the urn after n prints. Show that (Xn)n∈N is a Markov chain.

2- Give its associated and reduced graph and its transition matrix. Is the chain homogeneous?

3- Classify the states.

4- What type of chain (absorbent, irreducible, ergodic) is it?

5- Give a possible trajectory of size 10.

6- Determine the expectation of the number of prints until the urn contains only the red ball.

7- Does the chain admit a stationary distribution? a borderline distribution? Calculate limP^(n), where 'n' tends towards infinity.


1- We denote that E=(1,2,3) and the process is discrete to a discrete state space the state of Xn+1 depends only on the previous state Xn,

Therefore Xn is a Markov chain.


  • Transition Matrix :

Transition matrix

  • Associated graph :

enter image description here

The chain is not homogeneous because p(2/3) is different from p(2/1).

3- Classify the states: state classification: we have three transitory states (class):

  • State 1 does not communicate with any state other than itself.
  • State 2 communicates with state 1 but state 1 does not communicate (same for state 3 ).

Hence: E={1}U{2}U{3}.


  • The chain is not ergodic because it does not admit a recurrent state.
  • The chain is not irreducible because it admits 3 classes.
  • The chain is absorbent because state 1 is an absorbent state and all non-absorbent states (2/3) reach state 1.

5- Trajectory of size 10 :

Trajectory of size 10


  • Can you please tell me if I'm correct and I'm doing good, if not please correct me.
  • Can anyone please guide me solve questions 5 and 6.