Question 1 Consider a Markov Chian {X (t)} with space X = {0, 1, 2, 3, 4} and tran- sition rate matrix
−λ λ 0 0 0
0 −(λ+μ1) μ1 λ 0 Q=0 0 −μ1 0 μ1 μ2 0 0 −(λ+μ2) λ
0 μ2 0 0 −μ2 (a) Draw the state transition rate diagram
(b) Write out the equations for state probability by using the flow balance method. Indicate which flow is into the state and out of the state.
(c) Determine the stationary state probabilities(if exists) for λ = 1, μ1 = 1/3 and μ2 = 7/4.
Question 2 Consider a pure death chain {X(t)}.(it is a special case of birth-and-death chainwithλj =0,forallj=0,···). Letλj =0andμj =jμ,j=1,2,···,N,whereN is a given number and μ is a fixed. Show that πj(t) = P(X(t) = j), j = 1,2,··· ,N, has a binomial distribution with parameters N and e−μt.
Question 3 Let {X(t),t ≥ 0} be a continuous time Markov Chain with the state space being X = {1,2,··· ,m}. For any i,j ∈ X and i ̸= j it has qij = 1. Compute Pij(t). (qij is the element of transition rate matrix located at i-th column and j-th row. Pij(t) is the transition function defined as Pij (t) = P(X (t) = j |X (0) = i). Hint: write out the CK equation and solve the equation ).
Question 4 The poisson process can be viewed as special case of Birth-and-Death
process. Let N(t) be the poisson process show that the state probability is P(N(t) =
k) = (λt)n e−λt if N(0) = 0 and λ is the birth rate of B&D process. (Hint: Derive the n!
above equation from the CK equation)
Question 5 In Chapter 5, we discussed a model called Linear growth model with immigration. Write a simulation of B& D process and verify that our derived growth function of the expected value M(t) is correct.(Hint: think about the method to estimate M(t)Simulate the sample path of M(t) and compute the sample mean of M(t).)
1