36 At the beginning of each day a piece of equipment is insp

36.)

At the beginning of each day, a piece of equipment is inspected to determine
its working condition, which is classified as state 1 = new, 2, 3, or 4 = broken.

We assume the state is a Markov chain with the following transition matrix: 1 0.95 0.05 0 2 0 4 0 0 0.9 0 0.1 0.875 0.125 (a) Suppose that a broken machine requires 3 days to fix. To incorporate this into the Markov chain we add states 5 and 6 and suppose that p(4, 5-1, p(5,6) 1, and p(6, 1) = 1. Find the fraction of time that the machine is working. (b) Suppose now that we have the option of performing preventative maintenance when the machine is in state 3 and that this maintenance takes 1 day and returns the machine to state 1. This changes the transition probab- ility to 1 0.95 0.05 0 2 0 0.9 0.1 0 0 Find the fraction of time the machine is working under this new policy

Solution