Consider a Markov chain Xn with state space 1 2 3 4 5 6 7 si

Consider a Markov chain X_n with state space {1, 2, 3, 4, 5, 6, 7} site 1 goes back to 1 or to 2, equally likely. site 2 goes to 1 or 3, equally likely. Site 3 goes to 2. Site 4 goes to 4 or 5, equally likely. Site 5 goes to 4. Site 6 goes to 3, 4, or 7, equally likely. Site 7 goes to 5 or 6, equally likely. Draw a graph for this chain. What are the communicating classes? Which sites are recurrent/transient? What are the periods? Starting at site 2, what is the expected number of steps needed to reach 3? Starting at site 6, what are the probabilities to end up in the various recurrent classes? (Recall that you need to collapse the recurrent classes to answer this question.) Answer (d), starting at site 7. Compute the limit as n right arrow infinity of P {X_n = 5|X_0 = x} for all seven sites x {1,...,7}.

Solution