Consider the random walk Markov chain whose transition proba

Consider the random walk Markov chain whose transition probaility matrix is given by

Starting in state 1, determine the mean time until absorption.

Solution

The mean for the morkov cahin is determined by

The expected number of transitions needed to change states is given by

? n?1 nq n

or 16 transitions X 0.3= 4.8

where q n   is the probability of changing states after n transitions. This requires that we do not change states for n?1 transitions and then change states, so

q n =p(1?p) n?1 .

So the mean would be given by