One property of the period of a state is that If state i has

One property of the period of a state is that: If state i has period d(i), then there exists an integer N depending on i such that for all integers n Greaterthanorequalto N, P_u^(n d(i)) > 0. Consider the Markov chain whose transition probability matrix is P = ||0 1 0 0 0 0 0 1 0 0 0.5 0 0 0.5 0 0 0 0 0 1 1 0 0 0 0|| Determine an integer N such that for all integers n Greaterthanorequalto N, P_00^(nd(0)) > 0.

Solution

On calculatoring some powers of the given matrix P, we can see that that first element becomes zero for 7th power for the last time. That is, for all the powers of the matrix greater than or equal to 8, that first element never becomes 0.

Therefore, for N=8, for all n>=8 we get P₀₀^(nd(0)) > 0.

Therefore, N=8