Markov chain calculation Please show steps Thank you Conside

Markov chain calculation

Please show steps.. Thank you!

Consider a car rental company with three branches in Indiana. These are the probabilities of returning the car on each of the branches, depending on where it was rented: Draw a graph representing the Markov chain that describes this system. Write the transition probability matrix P=[p_ij] - Obtain the steady-state probability vector pi=[pi_jj]; start by writing the system of linear equations to be solved. - Obtain the mean-recurrence-times m_ii; start by writing the equations to be solved. - Obtain the mean-first-passage-times for all West-Lafayette arrivals (first column). Start by writing the system of linear equations to be solved. - What is the mean sodium time for leaving each of the cities? - What is the probability of leaving Indianapolis in three steps? - What is the probability of leaving Columbus in three or less steps? - Given the following reward matrix: - Obtain the expected total reward for each city after one, two and three steps; start by writing the system of equations to be solved.

Solution