程序代写 1.[5] Consider a Markov chain with state space S. Show that for any x,y in

1.[5] Consider a Markov chain with state space S. Show that for any x,y in S
􏰃Pn(x,y) ≤ 􏰃Pn(y,y) n=0 n=0
where Pn(x,y) is the n−step transition function. (Hint: use equation (28) on page 14 of the text- book.)
Proof: The result is obvious if x = y. Next we assume that x ̸= y. By equation (28) in the textbook, we have that for any n ≥ 1

Copyright By PowCoder代写 加微信 powcoder

Pn(x,y) = 􏰃 Px(Ty = m)Pn−m(y,y).
This combined with the fact that P0(y,y) = 1 and P0(x,y) = 0 for x ̸= y implies that
􏰃Pn(x,y) = 􏰃􏰃Px(Ty =m)Pn−m(y,y)
n=1 m=1 ∞∞
= 􏰃Px(Ty=m)􏰃Pn−m(y,y) m=1 n=m
= P (T <∞)􏰃Pn(y,y)=ρ 􏰃Pn(y,y) xy xy ≤ 􏰃Pn(y,y) n=0 2. Let {Xn : n = 0,1,2,...} be a Markov chain with state space S = {1,2,3,4,5} and one-step transition probability matrix which is equivalent to 􏰃1 n=􏰃 =∞ n=1 p1 · · · pn n=1 p 12000 33 21 12 000 P=00010. 11100 333 (a)[3] Find all transient, recurrent states and identify all irreducible sets. Solution: The recurrent states are 1, 2, 5 and the transient states are 3, 4. The sets {1, 2} and {5} are irreducible. (b) [3] Find ρ{1,2} (x) for x = 3, 4. Solution: Set ρ{1,2} (3) = U, ρ{1,2} (4) = V . Then we have which implies U = P(3,1)+P(3,2)+P(3,3)U+P(3,4)V =V V = P(4,1)+P(4,2)+P(4,3)U +P(4,4)V = 23 + 31 U U = V = 1. 3.[4] Let {Xn : n = 0,1,2,...} be a Markov chain with state space {0,1,2,...} and one step transition probability function  p y=x+1  1 − px − qx x ≥ 1, y = x − 1 y = x wherep>0,q>0,p+q<1,q0 =0. ShowthattheMarkovchainisrecurrentifandonlyifq≥p. Proof: By equation (68) on page 33 of the textbook, the Markov chain is recurrent if and only ∞ q ···q ∞ 􏰀q􏰁n 4. [6] Question 26 in the Exercises of Chapter 1 in the textbook. (Note: γx in the textbook corre- sponds to Γx in class lecture note.) Proof: (a) For x > 0, we have
lim Px{T0 < Tn} y=x lim 􏰂n−1 n→∞ y=0 γy 􏰂x−1 γy which is 1 if 􏰂∞y=0 γy = ∞. (b) If 􏰂∞y=0 γy < ∞, then we have y=0 1−lim􏰂n−1 n→∞ y=0 γy 􏰂x−1 γy 1 − 􏰂∞y=0 γy lim Px{T0 < Tn} y=x lim 􏰂n−1 n→∞ y=0 γy 􏰂∞y=x γy 􏰂∞ γ 5. [4] Question 33 in the Exercises of Chapter 1 in the textbook. Solution: In this case we have P(ξ1 =0)=P(ξ1 =3)= 12. where 0 ≤ t0 < 1 satisfies Φ(t0) = t0. By direct calculation, we have Φ(t)−t= 12(t−1)(t2 +t−1)=0. By Theorem 3 of Lecture 9, the probability of extinction is μ = 32 = 1 . 5 > 1 . ρ = t0
Solving the equation we obtain
t0= 5−1=ρ.

6. [5] Question 34 in the Exercises of Chapter 1 in the textbook. Solution: Let q = 1 − p. Then we have
Φ(t) = 􏰃 pqxtx x=0
μ = 􏰃xpqx x=1
= pq 􏰃(qx)′
Ifp≥1/2,thenμ≤1. Thusρ=1.
Now assume that p < 1/2. The solutions of the equation Φ(t) = p = t 1−qt t = 1 ± √1 + 4pq = 1 ± (1 − 2p) = 1 or p/q. 2q 2q Since 0 ≤ p/q = p/(1 − p) < 1, it follows from Theorem 3 of Lecture 9 that ρ=p. 程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com