ECON 61001: Lecture 2 Alastair R. Hall
The University of Manchester
Semester 1, 2020-21
Alastair R. Hall
ECON 61001: Lecture 2 1 / 23
Outline of today’s lecture
Statistical properties of OLS
mean
variance
sampling distribution
Confidence intervals
coefficients prediction
Alastair R. Hall
ECON 61001: Lecture 2 2 / 23
Parameter estimation: background
Within Classical statistics paradigm, the desired properties for an estimator, θˆT , of an unknown p × 1 parameter vector, θ0, are:
unbiasedness: E[θˆT ] = θ0.
efficiency: Var [θˆT ] is an efficient (unbiased) estimator of θ0 iff
Var[θ ̃T] − Var[θˆT] = p.s.d. where θ ̃T is any other unbiased estimator of θ0.
Alastair R. Hall
ECON 61001: Lecture 2 3 / 23
Linear regression model
Recall that our model is:
where
CA1: true model is: y = Xβ0 + u.
CA2: X is fixed in repeated samples. CA3: X is rank k.
CA4: E[u] = 0.
CA5: Var[u]=σ02IT.
CA6: u ∼ Normal.
y = Xβ0 + u
Alastair R. Hall
ECON 61001: Lecture 2 4 / 23
OLS estimator
OLS estimator:
βˆT = (X′X)−1X′y.
Want to derive sampling distribution of βˆT . Will do this in stages,
deriving first: mean, E[βˆT].
variance, Var [βˆT ].
To this end, we substitute for y using CA1 then: βˆT =β0+(X′X)−1X′u.
Alastair R. Hall
ECON 61001: Lecture 2 5 / 23
Mean
E [ βˆ T ] = E β 0 + ( X ′ X ) − 1 X ′ u . From CA2, this expectation can be written as:
E[βˆT] = β0 + (X′X)−1X′E[u]. So, using CA4, we have:
E[βˆT] = β0. ⇒ βˆT is an unbiased estimator of β0.
Alastair R. Hall
ECON 61001: Lecture 2 6 / 23
Variance
ˆ ˆ ˆ ˆ ˆ ′ Var[βT] = E βT −E[βT] βT −E[βT] .
Using E[βˆT] = β0, formula for βˆT and CA2, it follows that: Var[βˆT] = E (X′X)−1X′uu′X(X′X)−1 ,
= (X′X)−1X′ E[uu′]X(X′X)−1.
From CA4 and CA5, it follows that:
Var[βˆT] = (X′X)−1X′σ02ITX(X′X)−1 = σ02(X′X)−1
Alastair R. Hall
ECON 61001: Lecture 2 7 / 23
Gauss-Markov Theorem
Under assumptions CA1 – CA5, OLS is the Best Linear (in y) Unbiased Estimator (BLUE) of β0 in the sense that
Var[β ̃T ] − Var[βˆT ] = p.s.d
where β ̃T is any other linear (in y) unbiased estimator of β0.
Proof: Letβ ̃T =Dy whereD=(X′X)−1X′+C forsomek×T matrix of constants C . Note that E [β ̃T ] = β0 implies CX = 0.
Using similar arguments to OLS,
Var[β ̃T] = σ02DD′ = σ02{(X′X)−1 + CC′},
and so,
which is psd by construction.
Var[β ̃T]−Var[βˆT] = σ02CC′
Alastair R. Hall
ECON 61001: Lecture 2 8 / 23
Sampling distribution
Recall
βˆT =β0+(X′X)−1X′u
so, from CA2 + CA6, βˆT is linear combination of rv’s with Normal
distribution, and so via Lemma 2.1 (Lecture Notes) βˆT ∼Nβ0,σ02(X′X)−1.
Alastair R. Hall
ECON 61001: Lecture 2 9 / 23
Sampling distribution: example
Example from video on Sampling distributions (with T = 5).
Alastair R. Hall
ECON 61001: Lecture 2 10 / 23
Sampling distribution: example
Which in this case looks just like Napoleon’s hat!
Alastair R. Hall
ECON 61001: Lecture 2 11 / 23
Sampling distribution: example
Shape better revealed by contour plot in which each ring connects points with same pdf value.
Alastair R. Hall
ECON 61001: Lecture 2 12 / 23
Sampling distribution: example
ˆ
Var [βT ] =
=
Var[βˆT,1] Cov [βˆT ,1 , βˆT ,2 ]
85.02 −5.29 −5.29 0.39
Cov[βˆT,1,βˆT,2] Var [βˆT ,2 ] ,
.
Alastair R. Hall
ECON 61001: Lecture 2 13 / 23
Inference
Recall that under Assumption CA1-CA6:
βˆT ∼Nβ0,σ02(X′X)−1.
To use this result for inference, we need an estimator of σ02.
OLS estimator is:
σˆT2 = e′e T−k
We now show E[σˆT2 ] = σ02.
Alastair R. Hall
ECON 61001: Lecture 2 14 / 23
Impact of estimation of σ02
Consider inference about β0,i based on βˆT,i. We have
βˆT,i ∼ Nβ0,i,σ02mi,i,
where mi,i is the ith main diagonal element of (X′X)−1, and so
βˆ T , i − β 0 , i
σ √m ∼ N ( 0, 1) .
0 i,i
If we replace σ0 by σˆT then we βˆ T , i − β 0 , i
σˆ √m ∼ Student’s t distribution with T-k df T i,i
Alastair R. Hall
ECON 61001: Lecture 2 15 / 23
Example
Example from video on Sampling distributions (with T = 5). Simulated sampling distributions:
Alastair R. Hall
ECON 61001: Lecture 2 16 / 23
Example
Simulated sampling distribution of t-statistics
Alastair R. Hall
ECON 61001: Lecture 2 17 / 23
Example
Comparison of Student’s t distribution with 3 df to standard normal distribution.
Alastair R. Hall
ECON 61001: Lecture 2 18 / 23
Confidence interval for β0,i
A 100(1 − α)% confidence interval for β0,i is: βˆ ±τ (1−α/2)σˆ√m
where τT −k (1 − α/2) is 100(1 − α/2)th percentile of Student’s t distribution with T − k df.
T,i T−k T
i,i
Alastair R. Hall
ECON 61001: Lecture 2 19 / 23
Example: traffic fatalities in CA
From Lecture 1:
yˆt = controls − 0.030 ∗ beltt + 0.0671 ∗ mpht What do βˆbelt and βˆmph tell us about βbelt,0 and βmph,0?
Variability of estimator is: s.e.(βˆbelt) = 0.023
Leads to 95% confidence interval for βbelt,0: (−0.076,0.017)
Variability of estimator is: s.e.(βˆmph) = 0.021
Leads to 95% confidence interval for βmph,0: (0.026,0.108)
Alastair R. Hall
ECON 61001: Lecture 2 20 / 23
Prediction
Suppose we know xT+1 but not yT+1. (We assume model satisfies CA1-CA6 for t = 1, 2 . . . T + 1).
Then natural predictor of yT+1 is:
yp =x′ βˆ
So
T+1
T+1 T+1 T+1 0 T+1 =u −x′ (βˆ−β)
T+1 T
T+1 T+1 T
and prediction error is:
ep =y −yp =x′ β+u −x′ βˆ
T+1 T+1T 0
ep ∼ N0,σ2(1+x′ (X′X)−1x T+1 0 T+1
) T+1
Alastair R. Hall
ECON 61001: Lecture 2 21 / 23
Prediction
This leads to the 100(1 − α)% prediction interval for yT+1:
yp ± τ (1−α/2)σˆ (1 + x′ (X′X)−1x T+1 T−k T T+1
Example:
Suppose wish to predict fatalities for Jan 1990
(weekends= 12) if unem = 5: yp = 0.754122
T+1
))
T+1
95% prediction interval for y1990.1 is: (0.629, 0.879).
Alastair R. Hall
ECON 61001: Lecture 2 22 / 23
Further reading
Notes: Sections 2.3-2.6
See Greene:
Mean, Variance – Sections 4.3.1, 4.3.4, C.5.1
Gauss-Markov Theorem – Section 4.3.5 Confidence Intervals, Section 4.5.1 Prediction, Section 4.6.
Alastair R. Hall
ECON 61001: Lecture 2 23 / 23