A Markov chain {s}... with state space N={1, 2, 3} has a sequence of realizations and process transitions as follows: Transition (1,1) | (1,2) | (1,3) (2,1) | (2,2) S+1 _(2,3) (3,1) (3,2) (3,3) 2 1 1 1 1 1 3 1 3 3 3 4 3 2 1 3 3 2 7 2 2 8 2 2 1 3 10 3 2 11 2 12 1 2 13 2 1. 14 1 3 15 3 16 1 Number of frequencies : 1 1 2 3 2 1 3 1 91 92 93 a determine the transition frequency matrix 0- 2 021 °22 °23 (31 °32 °33, 1 A1 P12 P13 b. determine the estimated transition probability matrix| P= 2 p,, P22 P23 P31 P32 P33,

The sequence (Xn)n>o is a Markov chain with transition matrix 0 을 0 0 0 3 1 0 0 0 4 4 0 } 0 0 0 1 2 1 1 0 0 3 3 0 } 0 } 0 0 0 0 0 0 0 (a) Draw the transition diagram for (Xn). (b) Determine the communicating classes of the chain, stating whether each class is recurrent or transient. (c) Determine the period of each communicating class.

Recall that the graph of a discrete-time, finite-state, homogeneous Markov chain on two states 1, 2 with transition matrix is CO (: :) 1 T = If instead a new Markov chain on three states {1, 2, 3} has transition matrix 10 1 0 (a) Draw the corresponding graph for this new Markov chain. (b) Is the Markov chain with transition matrix T irreducible?

1. For each of the transition matrices given below, list absorbing state(s), and determine the period of each of the states in each Markov chain. (a) P = (d) P = 1120 CLI2L 1434L3 0 ܚ . ܟ ܝ. | ܟ ܝܕ . ܚ 0 00 0 0 0 0 0 3I4LI4 LIABI+ 0 0 (b) P = 1 0 Lo (e) P = 0 1 1 0 0 1333 COLITOLIN 0 0 0 0 0 213 100 0 0 3 00 100 0 -0 WIN 1 0 0 0 0 0 0 00 1 0 0 4|012 TWO O 0 0 0 0 0 (c) P = (f) P = 0 1|2 1|3 OLT20IN +15 O N|TN|TO 120 HINO OLI3 O 0 LI5L4L 1113 2I5

2. Consider a Markov chain {Xn}n≥o having the following transition diagram: 1/2 1 2 5 1/4 3 4 1/2 1/4 6 7 1/2 For this chain, there are two recurrent classes R1 = {6,7} and R2 = {1,2,5}, and one transient class R3 = {3,4}. (a) Find the period of state 3. (b) Find f33 and f22. (c) Starting at state 3, find the probability that the chain is absorbed into R1. (d) Starting at state 3, find the mean absorbation time, i.e., the expected number of steps that the chain is absorbed into R1 or R2. Note: there are missing transition probabilities for this chain, but no impact for your solution.

Please answer #28 and explain your reasoning.

A box contains 1 red pen and 2 blue pens. Every time, a pen is selected at random from the box and replaced with a pen of the opposite color, until all the pens in the box have the same color. Build the transition matrix O red 1 red 2red 3 red O red 1 red 2 red 3 red 1/2 1 1/3 2/3 1/4 || 3/4