P 12 13 16 0 13 23 12 0 12SolutionThe Markov property says t

P=     1/2      1/3      1/6
          0         1/3     2/3
         1/2       0        1/2

Solution

The Markov property says the distribution given past time only depends on the most recent time in the past.

1) $P(X_6=1 | X_4=4,X_5=1, X_0=4) = P(X_6 =1 | X_5 =1)$ which is the 1->1 transition entry (in position (1,1) which is $0.3$. The Markov property told us that the conditional property only depended on $X_5=1$.

2) $P(X_2 = 3, X_1 = 3 | X_0 = 1) = P(X_2 = 3 | X_1 = 3 , X_0 = 1) P(X_1 = 3 | X_0 = 1) = P(X_2 = 3 | X_1 = 3) P(X_1 = 3 | X_0 = 1)$ so this is the probability of transitioning from 3->3 times the probability of transitioning from 1->3. I don\'t know if your probabilities evolve from the left or right (i.e. if you left or right multiply by transition matrix for a probability vector, but once you know that you can look at either the (3,3) and (1,3) locations of the matrix or (3,3) and (3,1) entries of the matrix). We just used definition of conditional property and the Markov property here.