Problem 3 You have in your pocket You are given a coin that

Problem 3 You have in your pocket. You are given a coin that lands heads with probability p. Every minute you toss the coin once. If it is heads you gain an extra $1. If it is tails you lose a $1. If you reach $0 then you will have $0 for the rest of the game, regardless of what your coin comes up. The (small) House has $5. So if you reach $8 you will have $8 for the rest of the game regardless of what your coin lands. Is this a. Markov chain? Why or why not? If it is, write its transition matrix.

Solution

This is not a markov chain. Consider $0,$1 .....$8 to be the different states. It is clearly given that once you reach the stage $8 then you can\'t go anywhere else, which is not in the case of Markov chain.