Consider the Markov chain with transition matrix A Find the

Consider the Markov chain with transition matrix A = Find the transient and recurrent states. Find the mean time spent in 2 if we start from 1, as well as the mean time spent in 2 if we start from 2.

Solution

By looking at the transitions:

1-{1,2,3,4}, 2-{1,2,3}, 3-{3,4}, 4-{3}

1,2,3 is an closed irreducible set, so state 1, 2,3 are recurrent.

State 2 communicates with 1, which does not communicates with it, so state 2 is transient. State 3 communicates with 4, which does not communicates with it, so state 3 is transient.