4. Consider an aperiodic, irreducible Markov chain Xn (n = 0, 1, ...) with the N + 1 states 0, 1,..., N having stationary probabilities π¿ (i = 0, 1, ..., N). Suppose that the initial distribution of the Markov Chain is chosen according to the stationary probabilities. That is, P(X = i) = πi, i = 0, 1, ..., N. Define : Qij = P(X0 = j\X1 = i), Q = [Iij](N+1)×ばつ(N+1), 0 ≤ i, j ≤ N. (2) The square of the matrix Q is written as Q2 = [93] (N+1)x(N+1). Prove that q¡ = P(X0 = j|X2 = i). dj

a)

9. |Show that if X,, X2,... is a Markov chain, then it is statistically determined by its |second-order probability masses: P{X, = Xị , X, = x}, for any i, j = 1, 2,..

Let (X0, X1, X2, . . .) be the discrete-time, homogeneous Markov chain on state space S = {1, 2, 3, 4, 5, 6} with X0 = 1 and transition matrix

C)

Please answer #28 and explain your reasoning.