1.[5] Consider a Markov chain with state space S. Show that for any x,y in S
Pn(x,y) ≤ Pn(y,y) n=0 n=0
where Pn(x,y) is the n−step transition function. (Hint: use equation (28) on page 14 of the text- book.)
Proof: The result is obvious if x = y. Next we assume that x ̸= y. By equation (28) in the textbook, we have that for any n ≥ 1
Copyright By PowCoder代写 加微信 powcoder
Pn(x,y) = Px(Ty = m)Pn−m(y,y).
This combined with the fact that P0(y,y) = 1 and P0(x,y) = 0 for x ̸= y implies that
Pn(x,y) = Px(Ty =m)Pn−m(y,y)
n=1 m=1 ∞∞
= Px(Ty=m)Pn−m(y,y) m=1 n=m
= P (T <∞)Pn(y,y)=ρ Pn(y,y) xy xy
≤ Pn(y,y) n=0
2. Let {Xn : n = 0,1,2,...} be a Markov chain with state space S = {1,2,3,4,5} and one-step transition probability matrix
which is equivalent to
1 n= =∞ n=1 p1 · · · pn n=1 p
12000 33
21 12 000
P=00010. 11100 333
(a)[3] Find all transient, recurrent states and identify all irreducible sets.
Solution: The recurrent states are 1, 2, 5 and the transient states are 3, 4. The sets {1, 2} and {5} are irreducible.
(b) [3] Find ρ{1,2} (x) for x = 3, 4.
Solution: Set ρ{1,2} (3) = U, ρ{1,2} (4) = V . Then we have
which implies
U = P(3,1)+P(3,2)+P(3,3)U+P(3,4)V =V V = P(4,1)+P(4,2)+P(4,3)U +P(4,4)V
= 23 + 31 U
U = V = 1.
3.[4] Let {Xn : n = 0,1,2,...} be a Markov chain with state space {0,1,2,...} and one step transition probability function
p y=x+1
1 − px − qx
x ≥ 1, y = x − 1 y = x
wherep>0,q>0,p+q<1,q0 =0. ShowthattheMarkovchainisrecurrentifandonlyifq≥p.
Proof: By equation (68) on page 33 of the textbook, the Markov chain is recurrent if and only ∞ q ···q ∞ qn
4. [6] Question 26 in the Exercises of Chapter 1 in the textbook. (Note: γx in the textbook corre- sponds to Γx in class lecture note.)
Proof: (a) For x > 0, we have
lim Px{T0 < Tn}
y=x lim n−1
n→∞ y=0 γy x−1 γy
which is 1 if ∞y=0 γy = ∞.
(b) If ∞y=0 γy < ∞, then we have
y=0 1−limn−1
n→∞ y=0 γy x−1 γy
1 − ∞y=0 γy
lim Px{T0 < Tn}
y=x lim n−1
n→∞ y=0 γy
∞y=x γy ∞ γ
5. [4] Question 33 in the Exercises of Chapter 1 in the textbook. Solution: In this case we have
P(ξ1 =0)=P(ξ1 =3)= 12.
where 0 ≤ t0 < 1 satisfies
Φ(t0) = t0.
By direct calculation, we have
Φ(t)−t= 12(t−1)(t2 +t−1)=0.
By Theorem 3 of Lecture 9, the probability of extinction is
μ = 32 = 1 . 5 > 1 . ρ = t0
Solving the equation we obtain
t0= 5−1=ρ.
6. [5] Question 34 in the Exercises of Chapter 1 in the textbook. Solution: Let q = 1 − p. Then we have
Φ(t) = pqxtx x=0
μ = xpqx x=1
= pq (qx)′
Ifp≥1/2,thenμ≤1. Thusρ=1.
Now assume that p < 1/2. The solutions of the equation
Φ(t) = p = t 1−qt
t = 1 ± √1 + 4pq = 1 ± (1 − 2p) = 1 or p/q. 2q 2q
Since 0 ≤ p/q = p/(1 − p) < 1, it follows from Theorem 3 of Lecture 9 that ρ=p.
程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com