MAST30001 Stochastic Modelling
Tutorial Sheet 5
1. Fix p ∈ (0, 1). Consider the “Gambler’s ruin” chain on S = {0, 1, . . . , k} with
pi,i+1 = p and pi,i−1 = 1 − p for 1 ≤ i ≤ k − 1. Let mi := mi,{0,k} denote the
expected hitting time of {0, k} starting from state i.
(a) Show that if p = 1/2 then mi = i(k − i).
(b) Show that if p 6= 1/2 then
mi =
i
1− 2p
−
k
1− 2p
[
αi − 1
αk − 1
]
,
where α = (1− p)/p.
2. In a tiny politically divided village there are 12 villagers living in a 3×4 grid as
shown below. On each border, 5 always support party 1, and 5 always support
party 0 as shown below.
1 1 1 1
1 ? ? 0
0 0 0 0
The opinion of each of the two interior villagers is influenced by their neighbours
and fluctuates over time as follows: at each discrete time n ∈ N, one of the two
interior voters (chosen uniformly at random) adopts the political preference of one
of its 4 immediate neighbours (also chosen uniformly at random). Let Ln and Rn
be the opinions carried by the left and right interior villagers (as in the picture)
respectively.
(a) What is the state-space of the Markov chain Xn = (Ln, Rn)?
(b) Draw the transition diagram for the chain Xn.
(c) Is the chain Xn:
i. Irreducible?
ii. Aperiodic?
iii. Reversible?
(d) What is the limiting proportion of time that the left interior villager supports
party 1?
(e) Suppose that at time n = 1000000 a vote is taken (and the whole village votes).
Estimate the probability that party 1 wins the majority of votes, explaining
your reasoning.
3. Each morning at 8am, it is raining with probability rm ∈ (0, 1), and each afternoon
at 4pm it is raining with probability ra ∈ (0, 1) (both are independent of all previous
weather conditions). Suppose that your MAST30001 lecturer has 2 umbrellas, and
that he departs home for work at 8am each day and departs from work to home at
4pm each day (with a very short commute). Whenever it is raining at departure
time s/he takes an umbrella on the trip if there was one available at the departure
point. Find the long run proportion of trips for which it is raining on his departure
but he has no umbrella available.