Consider a finite-state Markov chain in which some given state, say state 1, is accessible from every other state. Show that the chain has at most one recurrent class R of states and state 1 ϵ R....

Consider a finite-state Markov chain in which some given state, say state 1, is accessible from every other state. Show that the chain has at most one recurrent class R of states and state 1 ϵ R. (Note that, combined with Exercise 4.2, there is exactly one recurrent class and the chain is then a unichain.)

Exercise 4.2

Show that every Markov chain with M <>