Student
number
Semester 2 Assessment, 2021
School of Mathematics and Statistics
MAST30001 Stochastic Modelling, Assignment 1
Submission deadline: 8 p.m. Friday 10 September
This assignment consists of 3 pages (including this page)
Instructions to Students
• If you have a printer, print the assignment.
Writing
• There are 2 questions with marks as shown. The total number of marks available is 40.
• *** Write instructions you want that are not auto-loaded here, each preceded by \item
• Write your answers on A4 paper. Page 1 should only have your student number, the
subject code and the subject name. Write on one side of each sheet only. Each question
should be on a new page. The question number must be written at the top of each page.
Scanning
• Put the pages in question order and all the same way up. Use a scanning app to scan all
pages to PDF. Scan directly from above. Crop pages to A4. Make sure that you upload
the correct PDF file and that your PDF file is readable.
Submitting
• Go to the Gradescope window. Choose the Canvas assignment for this assignment. Submit
your file. Get Gradescope confirmation on email.
©University of Melbourne 2021 Page 1 of 3 pages Do not place in Baillieu Library
MAST30001 Stochastic Modelling, Assignment 1 Semester 2, 2021
Question 1 (25 marks)
A DTMC (Xn)n∈Z+ with state space § = {1, 2, 3, 4} has the following transition matrix
P =
a 2/3 0 0
1/2 0 b 0
0 1/2 0 c
d 0 1/2 0
.
(a) Find the values of a, b, c, d.
(b) Draw the transition diagram for this chain.
(c) Find P(X5 = 4|X4 = 3).
(d) Find P(X5 = 4|X2 = 1, X4 = 3).
(e) Find P(X5 = 4|X3 = 1).
(f) Find P(X4 = 3|X0 = 1).
(g) Does the chain have any absorbing state(s)?
(h) Is the chain irreducible?
(i) Is the chain periodic? If so, what is the period?
(j) Is the chain transient, null-recurrent, or positive recurrent?
(k) Is the process reversible? Why, or why not?
(l) Find the long run proportion of time spent in state 4.
(m) Find the expected time to reach state 2, starting from state 1.
(n) Find the expected time to reach state 3, starting from state 1.
(o) If the initial distribution is (3/8, 3/8, 1/8, 1/8), does the limiting distribution exist? If so,
find it.
(p) Give a simple and accurate estimate for the matrix P 1000.
(q) Starting from state 2, find the probability of hitting state 4 before state 1.
Page 2 of 3 pages
MAST30001 Stochastic Modelling, Assignment 1 Semester 2, 2021
Question 2 (15 marks)
A betting game involves 3 players, that start the game with amounts of money $x, $y, $z (all
> 0) respectively. At each round n ∈ N of the game, one player (the giver) is chosen uniformly
at random to give some money to one of the other players (the receiver) chosen uniformly at
random (independent of previous rounds). If these two chosen players had $V and $W at the
beginning of the round, then the giver must give the receiver min{$V, $W}, and the round ends.
(For those of you who may be familiar with e.g. no limit poker, you can think of this as having
two players doing an “all in” bet in each round).
The first player to reach $0 in this game is called the loser. After a loser has been determined
the remaining two players continue until one of those two players has all the money. The player
with all of the money at the end is called the winner.
Let the amounts of money at time n (i.e. after n rounds) of the 3 players be Xn, Yn, and Zn
respectively (so X0 = x, Y0 = y, Z0 = z). Let T1 = inf{n ≥ 1 : min{Xn, Yn, Zn} = 0} and
T2 = inf{n ≥ 1 : max{Xn, Yn, Zn} = x + y + z}.
(a) Explain in words what these times T1 and T2 represent in this game.
(b) Show that E[T1] ≤ 2 and E[T2] ≤ 4.
(c) Using Martingale theory (see graduate course in probability) one can prove that in this
game E[XT2 ] = E[X0] = x (XT2 represents the amount of money of player 1 at the random
time T2). Use this fact to find the probability that Player 1 is the winner of the game.
(d) Find the probability that Player 1 is the loser of the game if (x, y, z) = (1, 2, 3).
(e) Find the probability that Player 1 is the loser if (x, y, z) = (12, 24, 36).
End of Assignment — Total Available Marks = 40
Page 3 of 3 pages