Faculty of Science
School of Mathematics and Statistics
Statistical Modelling and Computing
Lecture Notes written by
Copyright By PowCoder代写 加微信 powcoder
Term 1, 2022
1 Elements of probability
1.1 Comments about the course
1.2 Probability
1.3 Random variables and distributions (univariate) 1.4 Expectations, variances and correlations
1.5 Multivariate distributions
2 General inference problem 2.1 Measurement precision 2.2 Statistical Models
2.3 Inference problem
2.4 Goals in Statistical Inference
2.5 Statistical decision theoretic approach to inference
MATH3821: Page 1 / 756
3 Principles of data reductions and inference 3.1 Data reduction in statistical inference
3.2 Sufficent partition example
3.3 Sufficiency principle
3.4 Neyman Fisher factorization criterion
3.5 Sufficiency examples
3.6 Lehmann and Scheffe’s method for constructing a minimal sufficient partition
3.7 Minimal sufficient examples
3.8 One parameter exponential family densities
3.9 Generalization to a k- parameter exponential family
MATH3821: Page 2 / 756
3.10 Ancillary statistic and ancillarity principle
3.11 Ancillary examples
3.12 Maximum likelihood inference
3.13 Maximum likelihood estimation an introduction 3.14 Information and likelihood
4 Classical estimation theory
4.1 Cramer-Rao inequality
4.2 Comments on applying the CR Inequality in the search of the UMVUE
4.3 CRLB attainability examples
4.4 Which are the estimators that could attain the bound?
MATH3821: Page 3 / 756
4.5 Rao-Blackwell theorem
4.6 Uniqueness of UMVUE
4.7 Completeness of a family of distributions
4.8 Theorem of Lehmann-Scheffe
4.9 Examples finding UMVUE using Lehmann-Scheffe theorem
5 Likelihood inference and first order asymptotics 5.1 Why asymptotics
5.2 Convergence concepts in asymptotics
5.3 Consistency and asymptotic normality of MLE.
5.4 Additional comments on asymptotic properties of MLE. 5.5 Delta method
MATH3821: Page 4 / 756
6 Hypothesis testing 6.1 Motivation
6.2 General terminology in hypothesis testing
6.3 Fundamental Lemma of Neyman- Pearson
6.4 Comments related to the Neyman- 6.5 Simple H0 versus composite H1-the“simple case” 6.6 Composite H0 versus composite H1
6.7 Unbiasedness. UMPU α-tests.
6.8 Examples
6.9 Locally most powerful tests
6.10 Likelihood ratio tests
MATH3821: Page 5 / 756
6.11 Alternatives to the GLRT.
7 Order Statistics 7.1 Motivation
7.2 Multinomial distribution
7.3 Distributions related to order statistics
8 Higher order asymptotics 8.1 Motivation
8.2 Moments and cumulants
8.3 Asymptotic expansions
8.4 Extensions of the saddlepoint method
MATH3821: Page 6 / 756
9 Robustness and estimating statistical functionals.
9.1 Motivation. Basic idea of robustness
9.2 Robustness approach based on influence functions
9.3 Using the influence function in practice of robust inference.
10 Introduction to the bootstrap 10.1 Motivation
10.2 Nonparametric bootstrap
10.3 Parametric bootstrap
10.4 Numerical illustration
10.5 Bootstrap estimate of bias 10.6 The jackknife estimate of bias.
MATH3821: Page 7 / 756
10.7 Relation of bootstrap and jackknife.
10.8 Confidence intervals based on the bootstrap 10.9 Extensions of bootstrap outside i.i.d. setting
11 Concluding comments 11.1 The Final Exam
11.2 My advice
11.3 MyExperience survey 11.4 Conclusion
MATH3821: Page 8 / 756
Slide 1.1 / 1.76
1 Elements of probability
1.1 Comments about the course
1.2 Probability
1.3 Random variables and distributions (univariate) 1.4 Expectations, variances and correlations
1.5 Multivariate distributions
Chapter 1 : Elements of probability
MATH3821: Page 9 / 756
1.1 Comments about the course Slide 1.2 / 1.76
Professor of Statistics at the School of Mathematics and Statistics.
Born in Bulgaria
PhD in Statistics from Humboldt University, Berlin
Research interest is Statistical Inference, specifically nonparametric statistics. Other areas of interest:multivariate analysis, specifically latent variable models. Application domains: finance, risk evaluation and risk management.
Chapter 1 : Elements of probability MATH3821: Page 10 / 756
1.1 Comments about the course Slide 1.3 / 1.76
A face to the voice!
Chapter 1 : Elements of probability MATH3821: Page 11 / 756
1.1 Comments about the course Slide 1.4 / 1.76
Who are you?
What students do we have here today?
Chapter 1 : Elements of probability MATH3821: Page 12 / 756
1.1 Comments about the course
Slide 1.5 / 1.76
Telephone 90655376
Office Red Centre 1038 (first floor Red Centre) Web-page: https://research.unsw.edu.au/people/
professor-spiridon-ivanov-penev
Online consultations: 10-11am Wednesday from the moodle page of the course; individual consultations (if needed): appoint- ments to be made via email and conducted using Zoom.
For administrative problems, contact the Student Services Office (Mrs Markie Lugton, RC-3072,
Chapter 1 : Elements of probability MATH3821: Page 13 / 756
1.1 Comments about the course Slide 1.6 / 1.76
I will give four hours of lectures per week except for week 6.
Wednesday 18:00 – 20:00
Online BBC Online BBC
18:00 – 20:00
Chapter 1 : Elements of probability
MATH3821: Page 14 / 756
1.1 Comments about the course Slide 1.7 / 1.76
Tutorials and computer labs (using RStudio) for this course are flexible and will be held during the lectures. More precise information will be given during lectures.
Online materials: Further information, skeleton lecture notes, and other materials will all be provided on Moodle.
A set of tutorial exercises will be available on Moodle. These problems are for you to enhance your mastery of the course. Some of the problems will be done in lectures, but you will learn a lot more if you try to do them before class.
Chapter 1 : Elements of probability MATH3821: Page 15 / 756
1.1 Comments about the course Slide 1.8 / 1.76
Software used
We will use the R [R Core Team, 2017] software during the term. It is one of the most widely used software for Statistical computation and Graphics.
Install it on your laptop: https://cran.r-project.org
Next install RStudio, a nice Graphical User Interface to R: http: //www.rstudio.com/products/rstudio/download
Chapter 1 : Elements of probability MATH3821: Page 16 / 756
1.1 Comments about the course Slide 1.9 / 1.76
Computer laboratories
Computer laboratories (RC-G012 and RC-M020) are open 9-5 Monday- Friday on teaching days. RC-M020 has extended teaching hours (usually 8:30-9pm Monday-Friday, and 9-5 Monday-Friday on non- teaching weeks).
Chapter 1 : Elements of probability MATH3821: Page 17 / 756
1.1 Comments about the course Slide 1.10 / 1.76
Course aims
The aim of the course is to introduce the main ideas and principles behind parametric and non-parametric inference procedures.
Both frequentist and Bayesian perspectives will be discussed.
Estimation, confidence set construction and hypothesis testing are discussed within a decision-theoretic framework.
Both finite sample optimality and asymptotic optimality will be defined and discussed.
Computationally intensive methods such as bootstrap are dis- cussed and are compared to asymptotic approximations such as Edgeworth expansions and saddlepoint method.
Students will learn how to determine appropriate inference pro- cedure and to draw inferences using the chosen procedure.
Chapter 1 : Elements of probability MATH3821: Page 18 / 756
1.1 Comments about the course
Slide 1.11 / 1.76
Assessment Details
Assignment 1 Assignment 2 Mid-session test Final examination
End of week 4
End of week 9 Week 7 (30/03) Duration
(*) Mid-session test and Final examination will be time-released assignments with additional time provided to upload the solutions already included.
In all assessments marks will be awarded for correct working and appropriate explanations and not just the final answer.
Consult the course outline on Moodle for more details.
Late assignments will not be accepted.
Chapter 1 : Elements of probability MATH3821: Page 19 / 756
10% 10% 20% 60%
2 weeks 2 weeks 2 hours* 3 hours*
1.1 Comments about the course Slide 1.12 / 1.76
Skills to be developed
Learn how statistical inference arises from the first principles of probability theory;
Learn the fundamental principles of inference: sufficiency, likeli- hood, ancillarity, and equivariance;
Learn the concepts of finite-sample and asymptotic efficiency of an inference procedure;
Master the parametric and non-parametric delta method, asymp- totic normality, Edgeworth expansions and saddlepoint method; Be able to estimate key population parameters of interest, to test hypotheses about them and to construct confidence regions; Be able to use in practice the parametric, nonparametric, Bayes and robust inference;
Learn how to use the computer package R to generate output for the most common inference procedures and for computer- intensive calculations such as bootstrapping and robust estima-
Chapter 1 : Elements of probability MATH3821: Page 20 / 756
1.1 Comments about the course Slide 1.13 / 1.76
My philosophy for MATH5905
Lecture notes provide a brief reference source for this course. At this stage, these are skeleton lecture notes only. Throughout the course other materials and textbooks will be used for deeper understanding.
New ideas and skills are first introduced and demonstrated in lectures, then students develop these skills by applying them to specific tasks in tutorials and assessments.
Computing skills will be used to some extent but this is not a course in computing; the computing part is mainly used to illustrate the theory/methodology.
Chapter 1 : Elements of probability MATH3821: Page 21 / 756
1.1 Comments about the course Slide 1.14 / 1.76
Any questions?
Please feel free to interrupt me at any time!
Chapter 1 : Elements of probability MATH3821: Page 22 / 756
1.2 Probability Slide 1.15 / 1.76
1.2 Probability
These lecture notes were originally written and developed by Professor .
Standard univariate distributions like binomial, Poisson, normal, Cauchy, logistic, exponential are assumed to be known and are summarised in the Table of Common Distributions on pages 621–626 of CB. A copy can be found on Moodle.
The revision mainly the follows the sections of the CB reference.
Chapter 1 : Elements of probability MATH3821: Page 23 / 756
1.2 Probability Slide 1.16 / 1.76
1.2.1 Events and probabilities
An experiment that includes randomness can be modelled with prob- abilities. An event A, for example: It will be raining tomorrow is assigned a probability, P(A), which is a number between 0 and 1. Here, the certain event has probability 1, while the impossible event has probability 0.
In the simplest probabilistic model, there is a finite number m of possibilities (often called outcomes) and each of them has the same probability 1/m.
Furthermore, a collection of k outcomes, where k is less than or equal to m, is called an event A and its probability is calculated as k/m. That is:
P(A) = the number of outcomes in A = k . the total number of outcomes m
Chapter 1 : Elements of probability
MATH3821: Page 24 / 756
1.2 Probability Slide 1.17 / 1.76
Example 1.1
Suppose there are n people in a Zoom meeting.
i) Find the probability that at least two people have the same
ii) Calculate the probability for n = 22.
iii) Calculate the probability for n = 23. Solution:
(i) Let An be the event that at least two people have the same birthday. The number of outcomes not in the event An is
k = 365×364×···×(365−n+1).
Chapter 1 : Elements of probability MATH3821: Page 25 / 756
1.2 Probability Slide 1.18 / 1.76
The total number of possible outcomes in the sample space of all birthday combinations is
m = 365×365×…365 = 365n. The probability that all birthdays are distinct is
P(Acn) = k = 365×364×···×(365−n+1). m 365n
Hence the probability that two or more people have the same birthday in the Zoom meeting is
P(An) = 1− 365×364×···×(365−n+1). 365n
Chapter 1 : Elements of probability
MATH3821: Page 26 / 756
1.2 Probability
Slide 1.19 / 1.76
ii) For n = 22 this probability is
1 – prod(365:(365 – n + 1))/365^n
#> [1] 0.4756953
that is P(A22) = 0.48.
iii) For n = 23 this probability is
1 – prod(365:(365 – n + 1))/365^n
#> [1] 0.5072972
that is P(A23) = 0.51. Chapter 1 : Elements of probability
MATH3821: Page 27 / 756
1.2 Probability Slide 1.20 / 1.76
Exercise 1.1 (revision)
Suppose there are n people in a Zoom meeting.
i) Find the probability that at least one person has the same birthday
ii) Find the value of n, that is the number of people needed in the Zoom meeting, so that the probability that at least one person has the same birthday as you approaches 21 .
Chapter 1 : Elements of probability MATH3821: Page 28 / 756
1.2 Probability Slide 1.21 / 1.76
1.2.2 Conditional probability and independence
We want to define conditional probabilities P(A|B), which are calcu- lated by updating the probability P(A) of a particular event under the additional information that a second event B has occurred.
Such conditional probabilities can be calculated as follows:
P(A|B)= P(A∩B), P(B)
where P(A ∩ B) is the probability that both A and B occur. Additionally, when events A and B are independent then
P(A ∩ B) = P(A)P(B) and P(A|B) = P(A). Chapter 1 : Elements of probability
MATH3821: Page 29 / 756
1.2 Probability Slide 1.22 / 1.76
Example 1.2
Four cards are dealt from the top of a well-shuffled deck of 52 playing cards. Find the probability that all four cards are aces.
We can calculate this probability by the methods of the previous section. The number of distinct groups of four cards is
4 = 270725
Only one of these groups consists of the four aces and every group is equally likely, so the probability of being dealt all four aces is
Chapter 1 : Elements of probability
MATH3821: Page 30 / 756
1.2 Probability Slide 1.23 / 1.76
Another way to calculate this probability is to first consider the probability that the first card is an ace, which is 4/52. Given that the first card is an ace, the probability that the second card is an ace is 3/51. Continuing this argument, we obtain:
4×3×2×1= 1 52 51 50 49 270725
Chapter 1 : Elements of probability
MATH3821: Page 31 / 756
1.2 Probability Slide 1.24 / 1.76
Example 1.3
Let us now see how the conditional probabilities change given that some aces have already been drawn. Four cards will again be dealt from a well-shuffled deck, and we now calculate:
P(4 aces in 4 cards |i aces in i cards), i = 1,2,3
The event“4 aces in 4 cards”is a subset of the event“i aces in i cards”.
Hence, from the definition of conditional probability we have:
P(4 aces in 4 cards |i aces in i cards)
= P({4 aces in 4 cards }∩{i aces in i cards})
P(i aces in i cards) = P(4acesin4cards)
P(i aces in i cards)
Chapter 1 : Elements of probability
MATH3821: Page 32 / 756
1.2 Probability Slide 1.25 / 1.76
The numerator has already been calculated and the denominator can be calculated using a similar argument.
P(iacesinicards)= k = (4i) m (52)
Hence, the conditional probability is
(52) P(4 aces in 4 cards |i aces in i cards) = i
(52)(4) 4i
= (4 − i)!48! (52 − i)!
Chapter 1 : Elements of probability
MATH3821: Page 33 / 756
1.2 Probability Slide 1.26 / 1.76
For i = 1, 2 and 3 the conditional probabilities are 0.00005, 0.00082 and 0.02041.
1/choose(52 – i, 4 – i)
#> [1] 4.801921e-05
1/choose(52 – i, 4 – i)
#> [1] 0.0008163265
1/choose(52 – i, 4 – i)
#> [1] 0.02040816
Chapter 1 : Elements of probability MATH3821: Page 34 / 756
1.2 Probability Slide 1.27 / 1.76
1.2.3 Bayes’ Theorem
Consider the following two equations that arise from the definition of the conditional probability:
P(A ∩ B) = P(A|B)P(B) and P(A ∩ B) = P(B|A)P(A) Equating and rearranging these two formulas gives Bayes’ theorem:
P(A|B) = P(B|A)P(A) P(B)
Chapter 1 : Elements of probability
MATH3821: Page 35 / 756
1.2 Probability Slide 1.28 / 1.76
Additionally, for a general partition A1, A2, . . . , An of the sample space S with P(Ai) > 0 for all i = 1,…,n, we have
P(B|Aj)P(Aj) P(Aj|B) = ni=1 P(B|Ai)P(Ai)
for each j = 1, . . . , n which follows from the law of total probability n
P(B|Ai)P(Ai).
Chapter 1 : Elements of probability
MATH3821: Page 36 / 756
1.2 Probability Slide 1.29 / 1.76
Example 1.4
When a coded telegraph message is sent, there are sometimes errors in transmission. In particular, Morse code uses ”dots”and ”dashes”, which are known to occur in the proportion of 3 : 4.
This means that for any given symbol:
P(dot sent) = 73 and P(dash sent) = 47.
Suppose there is interference on the transmission line, with probability 18 a dot is mistakenly received as a dash, and vice versa. If we receive
a dot, what is the probability that a dot was actually transmitted?
Chapter 1 : Elements of probability MATH3821: Page 37 / 756
1.2 Probability
Slide 1.30 / 1.76
From the information provided we have that
P(dot sent) = 73 and P(dash sent) = 47. Using Bayes’ theorem, we can write:
P(dot sent | dot received) = P(dot received | dot sent)
P(dot sent) . P(dot received)
Chapter 1 : Elements of probability MATH3821: Page 38 / 756
1.2 Probability Slide 1.31 / 1.76
Additionally, we can write
P(dot received) = P(dot received ∩ dot sent) + P(dot received ∩ dash sent) = P(dot received | dot sent)P(dot sent)+
P(dot received | dash sent)P(dash sent) =7×3+1×4
Combining these results, we have that the probability of correctly receiving a dot is
P(dot sent | dot received) = (7/8) × (3/7) = 21 . 25/56 25
Chapter 1 : Elements of probability
MATH3821: Page 39 / 756
1.2 Probability Slide 1.32 / 1.76
Exercise 1.2 (revision)
Suppose that 5% of men and 0.25% of women are colour-blind. A person is chosen at random and that person is colour-blind. Find the probability that the person is male. Assume males and females to be in equal numbers.
Chapter 1 : Elements of probability MATH3821: Page 40 / 756
1.2 Probability Slide 1.33 / 1.76
Exercise 1.3 (at lecture)
Two litters of a particular rodent species have been born, one with two brown-haired and one grey-haired (litter 1), and the other with three brown-haired and two grey-haired (litter 2). We select a litter at random and then select an offspring at random from the selected litter.
i) Find the probability that the animal chosen is brown-haired.
ii) Given that a brown-haired offspring was selected, find the proba-
bility that the sampling was from litter 1.
Chapter 1 : Elements of probability MATH3821: Page 41 / 756
1.3 Random variables and distributions (univariate) Slide 1.34 / 1.76
1.3.1 Random variables
In many experiments, it is easier to deal with a summary variable, a so-called random variable, than with the original probability structure.
A random variable X is defined as a function from a sample space S into the set of real numbers:
For example, consider an experiment where two dice are thrown, we
can define the random variable X as a sum of the numbers rolled.
Chapter 1 : Elements of probability MATH3821: Page 42 / 756
1.3 Random variables and distributions (univariate) Slide 1.35 / 1.76
1.3.2 Probability mass function (pmf) for discrete random variables
Let us now consider real-valued realisations x of a discrete random variable X. The probability mass function (pmf) of a discrete random variable X
f(x) = P(X = x),
describes the distribution of X by assigning probabilities for the events
Chapter 1 : Elements of probability MATH3821: Page 43 / 756
1.3 Random variables and distributions (univariate) Slide 1.36 / 1.76
1.3.3 Cumulative distribution function (cdf)
A cumulative distributi
程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com