Consider a Markov chain chain Xn with state space 1 2 3 4 5

Consider a Markov chain chain X_n with state space {1, 2, 3, 4, 5, 6, 7}. Site 1 goes back to 1 or to 2, equally likely. Site 2 goes to 1 or 3, equally likely. Site 3 goes to 2. Site 4 goes to 4 or 5, equally likely. Site 5 goes to 4. Site 6 goes to 3, 4, or 7, equally likely. Site 7 goes to 5 or 6, equally likely. Draw a graph for this chain. What are the communicating classes? Which sites are recurrent/transient? What are the periods? Starting at sit 2, what is the expected number of steps needed to reach 3? Starting at site 6, what are the probabilities to end up in the various recurrent classes? (Recall that you need to collapse the recurrent classes to answer this question.) Answer (d), starting at site 7. Compute the limit as n rightarrow infinity of P{X_n = 5|X_0 = x} for all seven sites x {1,...,7}.

Solution