Consider the random walk on the integers O 1 2 3 which takes

Consider the random walk on the integers {O, 1, 2, 3} which takes steps +1 (to the right) with probability 1/3 and -1 (to the left) with probability 2/3, except at the endpoints where there is reflection; this means that a step from 1 to 0 is always followed by a step from 0 to 1, and a step from 2 to 3 is always followed by a step from 3 to 2. Determine the transition matrix for this Markov chain. Calculate the stationary distribution of this Markov chain. Suppose the Markov chain has been running for a long time. What fraction of time has it spent in state 0

Solution

The transistion matrix is

The state {0} is closed communicating class and its fraction of time is 4(2/3)=8/3=2.667(in right direction) and 4(1/3)=4/3=1.33( in left direction).

	0	1	2	3
0	0	1/3	0	2/3
1	2/3	0	1/3	0
2	0	2/3	0	1/3
3	1/3	0	2/3	0