Consider a Markov chain with the following transition probab

Consider a Markov chain with the following transition probability matrix P = ||1/6 1/6 1/6 1/6 1/6 1/6 1/3 1/12 1/24 5/24 5/24 1/2 1/4 0 1/4 0 0 1/8 1/4 1/4 1/16 1/4 1/16 1/9 1/3 1/9 2/9 1/9 0 1/4 1/4 1/4 1/8 1/8|| Let state 0, 1, 3, 5 - be the acceptable states. Let states 2,4 - be unacceptable states. a. Find the rate of breakdown. b. Find the rate of repair.

Solution

First, let\'s agree on notation: by XnXn we mean the state at \"time\" nn; and we are told that the initial state (n=0n=0) is X0=vX0=v. The transition matrix, denotes Mi,j=P(Xn+1=j|Xn=i)Mi,j=P(Xn+1=j|Xn=i) (the row corresponds to the \'before\' state, the column to the \'after\' state).

For the first question, you want to compute a particular transition path. But remember that you start from X0X0 (it can help to draw a graph of the transitions), so you actually are computing:

P(X1=x,X2=z,X3=v|X0=v)==P(X1=x|X0=v)P(X2=z|X1=x)P(X3=v|X2=z)==0.6×0.9×0.7P(X1=x,X2=z,X3=v|X0=v)==P(X1=x|X0=v)P(X2=z|X1=x)P(X3=v|X2=z)==0.6×0.9×0.7

(The first equation is true because it\'s a Markov chain).

For the second, you need to compute the probabilities of arriving of each one of the five states at time n=4n=4. You could do that by summing all the paths that start from X0=vX0=v, but that would be painful. (In general, perhaps not so much in this case because there are few transitions with positive probability). A more elegant way is to recall that the 4-step transition probabilities is given by M4M4. Once you compute that, you just take the first row.