Consider an irreducible positive-recurrent Markov chain. Consider the renewal process {N jj (t);t 0} where, given X 0 = j, N jj (t) is the number of times that state j is visited from time 1 to t. For...

Consider an irreducible positive-recurrent Markov chain. Consider the renewal process {N_jj(t);t 0} where, given X₀
= j, N_jj(t) is the number of times that state j is visited from time 1 to t. For each i ≥ 0, consider a renewal-reward function R_i(t) equal to 1 whenever the chain is in state i and equal to 0 otherwise. Let π_i
be the time-average reward