ECON 61001: Lecture 1
Alastair R. Hall
The University of Manchester
Alastair R. Hall ECON 61001: Lecture 1 1 / 21
Outline of this lecture
Linear regression model
Model
Assumptions
Ordinary Least Squares
Sum of squares decomposition
Interpretation of OLS coefficients – the Frisch-Waugh-Lovell Theorem
Alastair R. Hall ECON 61001: Lecture 1 2 / 21
Examples of econometric models
assetreturns:R−Rf =β0(Rm−Rf)+error returns to education:
ln(w) = β0,1 + β0,2 ∗ed + β0,3 ∗exp + β0,4 ∗exp2 + error aggregate production function:
ln(Q) = β0,1 + β0,2 ∗ ln(L) + β0,3 ∗ ln(K) + error
change in inflation: ∆inf = β0,1 + β0,2 ∗ ∆inf (−1) + error
All have common structure: linear in the parameters, and additive error
Alastair R. Hall ECON 61001: Lecture 1 3 / 21
Data types and notation
Economic data typically comes in four types:
Cross-section – covered in course Time series – covered in course Panel data
Repeated cross-section
Notation for sample:
Cross-section: i = 1, 2, . . ., N
Times series: t = 1,2,…,T
For first part of course, results apply equally to both types of data
and use (default) of t notation.
When discuss large sample properties arguments are different and
will i or t notation as reminder of sample structure.
Alastair R. Hall ECON 61001: Lecture 1 4 / 21
Linear regression model
Wish to model relationship between yt (“dependent variable”) and k × 1 vector xt (“explanatory variables”).
Assume: and
yt =xt′β0+ut
yt, xt are observable but the error term ut is not.
β0 is an unknown k × 1 vector of “regression coefficients” (parameters).
Observe {yt,xt;t = 1,2…T} → estimate of β0.
Alastair R. Hall ECON 61001: Lecture 1 5 / 21
Linear regression model
More convenient to express model in matrix notation: y = Xβ0 + u
where
y is T ×1 with tth element yt X i s T × k w i t h t t h r o w x t′
u is T ×1 with tth element ut
Alastair R. Hall ECON 61001: Lecture 1 6 / 21
Classical assumptions
CA1: true model is: y = Xβ0 + u. CA2: X is fixed in repeated samples. CA3: X is rank k.
CA4: E[u] = 0.
C A 5 : V a r [ u ] = σ 02 I T . CA6: u ∼ Normal.
Implications for y:
y ∼ N ( X β 0 , σ 02 I T ) .
Alastair R. Hall
ECON 61001: Lecture 1
7 / 21
Estimation problem
Consider here estimation of β0 based on sample (y,X) using Ordinary Least Squares (OLS).
Define u(β) = y − X β and note u( · ) : S × B → RT . OLS minimand is:
T
QT (β) = u(β)′u(β) = (yt − xt′β)2
t=1
(Note: QT :S×B→[0,∞).) OLS estimator of β0 is:
βˆT = argminβ∈BQT (β)
Alastair R. Hall ECON 61001: Lecture 1 8 / 21
OLS
First order conditions (FOC):
∂QT(β)β=βˆ =0
∂β T Second order conditions (SOC):
∂2QT (β) ˆ = positive definite (p.d.) ∂β∂β′ β=βT
Alastair R. Hall ECON 61001: Lecture 1 9 / 21
OLS
We have
and so from Lemma 2.2(i)-(ii)[LN p.12]:
QT(β) = y′y − 2y′Xβ + β′X′Xβ ∂QT(β) = −2X′y + 2X′Xβ
∂β
and from Lemma 2.2(iii)
∂2QT(β) = 2X′X
∂β∂β′
F O C → X ′ ( y − X βˆ T ) = 0 a n d s o u s i n g C A 3 ,
βˆT = (X′X)−1X′y
and (using CA3) SOC satisfied.
Alastair R. Hall ECON 61001: Lecture 1 10 / 21
Model involves decomposition of y:
y = E[y] + u
OLS affects a similar decomposition:
y = yˆ + e
where
yˆ = XβˆT, vector of predicted values for y.
e = y − X βˆ T , v e c t o r o f O L S r e s i d u a l s . Note FOC ⇒ X′e = 0 and so
yˆ′e = 0
Alastair R. Hall ECON 61001: Lecture 1 11 / 21
OLS affects a similar decomposition of the variation of y in models that include an intercept. So now set: X = [ιT , X2].
The decomposition of the variation of y is as follows: TSS = ESS + RSS
where:
TSS = “Total sum of squares” = Tt=1(yt − y ̄)2 ESS = “Explained sum of squares” = Tt=1(yˆt − y ̄)2 RSS = “Residual sum of squares” = Tt=1 et2 = e′e.
This leads to the multiple correlation coefficient, R2: R2 = ESS
TSS
which is proportion of variation in y explained by linear regression on X.
Alastair R. Hall ECON 61001: Lecture 1 12 / 21
The OLS coefficients
We now develop a useful interpretation of OLS coefficients based on the Frisch-Waugh-Lovell (FWL) Theorem.
To present the FWL Thm we need to partition the regressors and coefficient vector conformably:
X =(X1,X2), β0=β1(k1×1), (T×k) (T×k1) (T×k2) β2 (k2×1)
and write model as
y = Xβ0 + u = X1β1 + X2β2 + u
Let βˆT,2 be the OLS estimator of β2 in this model. Alastair R. Hall ECON 61001: Lecture 1
13 / 21
FWL Thm
Now consider the alternative strategy for estimation of β2 ( here x2,l denotes the lth column of X2).
Step 1: Regress y on X1 via OLS and denote the associated vector of OLS residuals by w.
Step 2: For each l = 1, 2, . . ., k2, regress x2,l on X1 via OLS and denote the associated vector of OLS residuals by dl.
Step 3: Regress w on D, where D = (d1,d2,…,dk2), via OLS and denote the resulting vector of coefficient estimators by bˆ that is,
bˆ = (D′D)−1D′w.
Alastair R. Hall ECON 61001: Lecture 1 14 / 21
FWL Thm
F W L T h e o r e m : βˆ T , 2 = bˆ . Implications of FWL:
Consider case where X1 = [x1,x2,…,xk−1], and X2 = xk.
w and D = d represent the parts of y and xk that cannot be
linearly explained by X1.
Step 3 captures the relationship between y and xk once they have both been purged of any linear dependence they have on X1.
Alastair R. Hall ECON 61001: Lecture 1 15 / 21
FWL Thm
bˆ = 0 ⇒ any relationship between y and xk can be accounted for by their joint dependence on X1.
bˆ ̸= 0 ⇒ y and xk are linearly related in a way that cannot be explained purely by their joint dependence on X1.
βˆk captures partial effect = the unique contribution (relative to the other regressors in the model) of xk to the (linear) explanation of y.
Terminology:
Steps 1 and 2 are often referred to as “partialling out” the
effect of X1.
The regression in Step 3 is said to capture the relationship between y and xk controlling for X1.
Alastair R. Hall ECON 61001: Lecture 1 16 / 21
Example
In mid-1980’s there were two changes to federal highway regulations in US:
Jan 1986: seat belt law passed
May 1987: states allowed to raise highway speed limit from 55mph to 65mph
McCarthy (1994) investigates whether these changes affected the number of traffic fatalities in California.
Analysis uses: monthly data, Jan 1981 – Dec 1989.
Alastair R. Hall ECON 61001: Lecture 1 17 / 21
Example
Dependent variable, yt : % of highway accidents that resulted in one or more fatality.
Explanatory variables:
beltt : dummy variable equal to 1 for t ≥ 1986.1
mpht : dummy variable equal to 1 for t ≥ 1987.5
Alastair R. Hall ECON 61001: Lecture 1 18 / 21
Example
yˆt = 0.914 − 0.064 ∗ beltt
yˆt = 0.893 − 0.024 ∗ mpht
yˆt = 0.914 − 0.102 ∗ beltt + 0.057 ∗ mpht
Now introduce controls:
Linear time trend and monthly dummies:
yˆt = controls − 0.014 ∗ beltt + 0.078 ∗ mpht Plus state unemployment rate and number of weekends in
month:
yˆt = controls − 0.030 ∗ beltt + 0.0671 ∗ mpht
Alastair R. Hall ECON 61001: Lecture 1 19 / 21
However..
Sometimes the inclusion of controls can undermine the inference of interest.
Example: impact of climate variable, Ci , on economic activity, yi , based on cross-sectional country data
yi = α + γCi + error,
Suppose include controls (institutional measures, population etc)
yi = α + γCi + zi′δ + error,
If controls depend on climate then their inclusion masks the impact of climate on economic activity → problem known as over-controlling, (Dell, Jones & Olken, 2014).
Alastair R. Hall ECON 61001: Lecture 1 20 / 21
Further reading
See:
Lecture Notes Sections 1.1-1.3, 2.1-2.3
Greene
Linear regression model – Chapter 2 (Discussion of assumptions more general than Lecture 1 but does match Lecture 3)
OLS – Chapter 3 (Material in Section 3.4 not covered in lecture but read for concept of partial regression)
Alastair R. Hall ECON 61001: Lecture 1 21 / 21