1. A Markov chain has state space Z = {0, 1, 2, 3, 4} and transition matrix (1) Find the essential and inessential states. (2) Find the recurrent and transient states. 2. Determine the stationary...

1. A Markov chain has state space Z = {0, 1, 2, 3, 4} and transition matrix

(1) Find the essential and inessential states.

(2) Find the recurrent and transient states.

2. Determine the stationary distribution of the random walk considered in example 4.8 on condition p_i
= p, 0 <>