ISE 3414 – Stochastic Modeling and Analysis (P.O.R.)
Midterm Exam I
Fall 2020 – Prof. M. R. Taaffe)
NAME:
STUDENT ID NUMBER:
SECTION: 10:10 a.m. 11:15 a.m.
This exam is open-book and open-notes, and use of MATLAB. You have one week to complete and upload the exam via Canvas (under the “Practice Quiz” Section of the “Quizzes” page.
There are a total of 100 points.
PLEASE READ ALL QUESTIONS VERY CAREFULLY! You should upload only ONE .pdf file.
Write out complete answers, including fully specified algorithms and all numbers filled in the various vectors and matrices, to all questions. Also provide actual numbers for all questions. Use MATLAB for doing your calculations unless you really enjoy doing arithmetic and matrix manipulations by hand. Include (for example, via “Snip and Paste” (Press Winkey + Shift + S)) MATLAB output for numerical results that you obtained via MATLAB. You do NOT have to include any code provided by the Instructor via the Canvas pages. Attach extra pages to your .pdf file if necessary to fit in your MATLAB computations.
IMPORTANT For the purposes of this Midterm, you may not state an answer in terms of P(∞), or P∞, or P(n∗) where n∗ is some sufficiently large number, unless you are specifically asked to do so. In other words you cannot state an answer in a manner that includes just raising the single-step transition matrix to a very large power, unless asked to do so.
The Virginia Tech Honor Code Pledge:
“I have neither given nor received unauthorized assistance on this take-home test. ”
Signature:
SCORE:
ISE 3414
1. (25 total points): Here are P, P2, and P3. In answering the following questions you do not have to provide a numerical answer; it will be sufficient to show an expression with all of the numbers in the right places, so that only simple arithmetic needs to be completed.
0.4 0.1 0.5 0.24 0.2 0.56 0.212 0.256 0.532 P= 0.3 0.6 0.1 , P2 = 0.31 0.41 0.28 , P3 = 0.275 0.333 0.392
0.1 0.2 0.7 0.17 0.27 0.56 0.205 0.291 0.504 (a) (5 points): What is the value of p(4)?
2,3
(b) (5 points): If the process starts in State 3, what is the expected number of visits to State 2 in the first 3 epochs?
Midterm I, Fall 2020 Page 2 of 14 Taaffe
ISE 3414
(c) (5 points): For this part, assume that the initial state is equally likely to be any one of the three states. What is the probability that the process is in State 3 at epoch 3. You need to write an expression with all of the numbers in the right places to show us how you computed your result.
(d) (5 points): What is the variance of the sojourn time for State 1?
(e) (5 points): What is the State-1 sojourn-time probability-mass function, and its associ- ated parameters? (State this completely and clearly).
Midterm I, Fall 2020 Page 3 of 14 Taaffe
ISE 3414
2. (10 points): Consider the following transition diagram. Write down a sufficient set of linear equations (preferably in matrix form) to solve for the mean first-passage time from State 2 to State 4. Solve the equations in MATLAB (or by hand if you do want to), but write the equations with specific numerical coefficients. You may use ugly simultaneous linear equations, explicitly, or you may write your answer in matrix-vector form, as long as every quantity or expression is defined. Include “Snip and Paste” MATLAB for MATLAB computations.
Figure 1: State Transition Diagram
Midterm I, Fall 2020 Page 4 of 14 Taaffe
ISE 3414
3. Compartmental Random Particle Movement Model
(35 total points) In a physics experiment a particle is placed in Chamber A of the space shown below. The scientists observe the length of time for the particle to first enter the special chamber, Chamber F , where time is measured by the number of chambers the particle visits. For example if the particle starts in Chamber A and proceeds to Chamber D and then to E andthentoF,thenthelengthoftimetogofromAtoF is3.
Assume that the particle will move in a random way bouncing off the walls the chamber that it is currently in until it randomly exits the current chamber through any available opening out of a chamber with equal likelihood. For example, if there are two openings available in a particular chamber then the particle will eventually exit this chamber by either of the openings with equal probability. (In counting the openings, include the one that the particle used to enter the chamber because the particle could retrace its steps.)
Note: When the particle enters Chamber F (or C) it will leave Chamber F (or C) via the only opening possible and re-enter Chamber E (B).
Figure 2: Compartmental Random Particle Movement Model
In this picture there is only an opening if there is a gap in the lines indicating a wall.
(a) (5 points) Model the particle’s journey through the system to the special chamber, Chamber F, as a Markov Chain and write the single-step transition matrix (complete with numbers). You must also write how you specify the initial state.
Midterm I, Fall 2020 Page 5 of 14 Taaffe
ISE 3414
(b) (5 points) Classify the states and state whether this is an ergodic Markov Chain or a Terminating Markov Chain.
For this problem do NOT use the FPT ALGORITHMIC form of a solution. Use vector/matrix methods to compute your answers.
(c) (5 points) Using specific and well-defined variables and expressions, describe how to compute the expected number of time steps (chambers visited) necessary to reach the special chamber, Chamber F, starting from Chamber A.
Midterm I, Fall 2020 Page 6 of 14 Taaffe
ISE 3414
(d) (10 points) Write two methods or algorithms to compute the long-run fraction of visits that the particle spends each of the chambers.
CAREFUL: Make sure that you account for possible periodicity in your analysis.
i. (5 points) Algorithm One: This algorithm should involve taking the P matrix to larger and larger powers. Do the work in MATLAB. Snip-and-Paste your MATLAB work here.
Midterm I, Fall 2020 Page 7 of 14 Taaffe
ISE 3414
ii. (5 points) Algorithm Two: This algorithm should involve a matrix inverse. Do the work in MATLAB. Cut and Paste your MATLAB work here.
Midterm I, Fall 2020 Page 8 of 14 Taaffe
ISE 3414
(e) (10 points) We further describe this Particle Model. In the revised system, whenever the particle enters Chamber A it becomes (or remains) a positively charged particle (+), regardless of its charge when entering Chamber A. Whenever the particle enters Chamber F it becomes (or remains) a negatively charged particle (−), regardless of its charge when entering Chamber F.
i. (5 points) Compute the pmf of time (as measured by number of visits) from when the particle becomes negatively charged (−) until it switches to becoming positively charged (+). Print or plot the this fpt pmf (only print the first 50 pmf values.
Midterm I, Fall 2020 Page 9 of 14 Taaffe
ISE 3414
ii. (5 points) Compute the variance of time from when the particle becomes negatively charged (−) until it switches to becoming positively charged (+).
Midterm I, Fall 2020 Page 10 of 14 Taaffe
ISE 3414
4. (30 total points): A player goes into a casino to gamble on roulette. Her objective is to have fun, not to make a lot of money, so she employs a simple strategy. She will get $50 in $10 chips (i.e., 5 chips), bet one chip at a time on either red or black according to her hunch at the time, and play until she either runs out of chips or has doubled her initial stake (i.e., has 10 chips), at which time she will declare herself to be a “winner,” and go home to share her winnings with her partner.
In roulette, there are 18 red, 18 black, and 2 green slots on the wheel. The ball is equally likely to land in any one of the slots. The “house” (the casino itself) wins all bets if the ball lands in a green slot.
(a) (5 points): Write out the single-step transition matrix for a Markov chain model that tracks the player’s gains and losses, arranged in the standard form (block-matrix form) for this kind of Markov chain. (You may want to start by drawing the transition diagram, but the question asks for the matrix.) Also describe in words the contents of each of the matrix blocks.
Midterm I, Fall 2020 Page 11 of 14 Taaffe
ISE 3414
(b) (5 points): Explain how to calculate the probability that our player ends the evening hav- ing doubled her original stake (as opposed to losing it all). (Do not do the calculations, but show how the equations that have to be solved and any additional transformations that might be necessary to answer the question).
Midterm I, Fall 2020 Page 12 of 14 Taaffe
ISE 3414
(c) (5 points): Assuming that a roulette wheel averages thirty plays per hour and that our player begins to play at 7:00 p.m., explain how to calculate the expected time that evening or the following day that she will finish. (Do not do the calculations, but show the equations that have to be solved and any additional transformations that might be necessary to answer the question.)
Midterm I, Fall 2020 Page 13 of 14 Taaffe
ISE 3414
(d) (5 points): Explain how to calculate the expected number of plays over the entire course of her session that she is ahead (i.e., she has more than the $50 that she started with). (Do not do the calculations, but show the equations that have to be solved and any additional transformations that might be necessary to answer the question.)
(e) (10 points): Write the algorithm compute the pmf for the duration of a winning evening (in terms of the number of wheel spins).
Midterm I, Fall 2020 Page 14 of 14 Taaffe