Markov Chains) (Choosing Balls from an Urn) An urn contains two unpainted balls at present. We choose a ball at random and flip a coin. If the chosen ball is unpainted and the coin comes up heads, we...

Markov Chains) (Choosing Balls from an Urn) An urn contains two unpainted balls at present. We choose a ball at random and flip a coin. If the chosen ball is unpainted and the coin comes up heads, we paint the chosen unpainted ball red; if the chosen ball is unpainted and the coin comes up tails, we paint the chosen unpainted ball black. If the ball has already been painted, then (whether heads or tails has been tossed) we change the color of the ball (from red to black or from black to red). To model this situation as a stochastic process, we define time t to be the time after the coin has been flipped for the t’th time and the chosen ball has been painted. The state at any time may be described by the vector [u r b], where u is the number of unpainted balls

in the urn, r is the number of red balls in the urn, and b is the number of black balls in the urn. Example: We are given that ?0 = [2 0 0]. After the first coin toss, one ball will have been painted either red or black, and the state will be either [1 1 0] or [1 0 1]. Hence, we can be sure that ?1 = [1 1 0] or ?1 = [1 0 1]. The transition matrix is given below:
a. Draw the Graphical Representation of Transition Matrix for Urn.
b. Transition Probabilities If Current State Is [1 1 0] are given below I the table as an
example:
Explain the computations of the probabilities given in the row when the Current State Is [1 1 0]. ( There are indeed 6 probabilities, 1⁄4, 1⁄4, 0,0,0, 1⁄2), Explain all six probabilities and how you compute them in words