MAST30001 Stochastic Modelling
Tutorial Sheet 2
1. Show that the Markov property does not in general imply that for any events A,B
and C,
P(Xn+1 ∈ C|Xn ∈ B,Xn−1 ∈ A) = P(Xn+1 ∈ C|Xn ∈ B).
(That is, define a Markov chain and events A,B,C where the equality doesn’t hold.)
2. Let (Xn)n∈Z+ be a Markov chain with state space {1, 2, 3} and transition matrix
0 1/3 2/31/4 3/4 0
2/5 0 3/5
(a) Draw the transition diagram corresponding to this chain.
(b) Compute P(X3 = 1, X2 = 2, X1 = 2|X0 = 1).
(c) If X0 is uniformly distributed on {1, 2, 3}, compute P(X3 = 1, X2 = 2, X1 = 2).
(d) Now assuming that P(X0 = 1) = P(X0 = 2) = 1/2, compute P(X1 = 1, X4 =
1, X7 = 1).
3. A simplified model for the spread of a contagion in a small population of size 4 is
as follows. At each discrete time unit, two individuals in the population are chosen
uniformly at random to meet. If one of these persons is healthy and the other has the
contagion, then with probability 1/4 the healthy person becomes sick. Otherwise
the system stays the same.
(a) If Xn is the number of healthy people at step n, then explain why X0, X1 . . .
is a Markov chain.
(b) Specify the transition probabilities of Xn.
(c) Draw the transition diagram for this chain.
(d) If initially the chance that a given person in the population has the disease
equals 1/2, determined independently, then what is the chance everyone has
the disease after two steps in the process?
(e) Now suppose that exactly one person is infected at time 0. Find the expected
time until everyone is infected.
4. Let (Yn)n≥0 be i.i.d. random variables with P(Yi = 1) = P(Yi = −1) = 1/2 and let
Xn = (Yn+1 + Yn)/2.
(a) Find the transition probabilities P(Xn+m = k|Xn = j) for m = 1, 2, . . . and
j, k = 0,±1.
(b) Show that (Xn)n≥0 is not a Markov chain.