ECON3350/7350
Introduction and Review of the Classical Linear Regression Model
Eric Eisenstat
The University of Queensland
Lecture 1
Eric Eisenstat (School of Economics) ECON3350/7350 Week 1
1 / 16
The Course
Course Coordinator: Eric Eisenstat
Room 523B, Colin Clark Building (39)
Email: e.eisenstat@uq.edu.au
Consultation Hours: Mon 14:00-16:00 or by appointment
Tutor consultation timetables will be available on Blackboard.
Lecture Slides, Recordings, Practicals and Assessment material on Blackboard
Be sure the visit the Blackboard site regularly for updates, announcements, etc.
Eric Eisenstat (School of Economics) ECON3350/7350 Week 1 2 / 16
The Course
Practicals using Stata The textbooks:
Enders, W., Applied Econometric Time Series, 4th Edition, Wiley, 2015.
Hill, R.C., Griffiths, W.E. and G.C. Lim, Principles of Econometrics, 4th edition, New Jersey: Wiley, 2011.
Verbeek, M. J. C. M., A Guide to Modern Econometrics, 3rd edition. Chichester: John Wiley and Sons, 2008.
Lectures: Concepts
Practicals: Consolidating concepts and computing using data (attending tutorials is highly recommended)
Assessment tasks: Different for ECON3350, ECON7350; please refer to
Blackboard
Eric Eisenstat (School of Economics) ECON3350/7350 Week 1
3 / 16
ECON3350 Assessment
Assignment Task Assignment 1 Mid Semester Exam Assignment 2 Final Exam
Due Date Weight 23 Mar 20 23:59 15% 6 Apr 20 16:00 30% 11 May 20 23:59 15% Examination Period 40%
Eric Eisenstat
(School of Economics)
ECON3350/7350
Week 1
4 / 16
ECON7350 Assessment
Assignment Task Research Report 1 Research Report 2 Final Exam
Due Date Weight 6 Apr 20 23:59 30% 26 May 20 23:59 30% Examination Period 40%
Eric Eisenstat (School of Economics)
ECON3350/7350
Week 1 5 / 16
Review of Classical Regression
Recommended readings
Author
Title
Chapter
Call No
Verbeek
A Guide to Modern Econometrics
2, 3, 4, 5
HB139 .V465 2012
Hill et al.
Principles of Econo- metrics
1-10
HB139 .H548 2011
Eric Eisenstat (School of Economics) ECON3350/7350 Week 1 6 / 16
The Classical Regression Model
A rudimentary approach in analyzing economic data is to specify a linear relationship:
yi =β1 +β2×2,i +···+βkxk,i +εi.
We refer to yi as the dependent variable, x2,i, . . . , xk,i as the regressors, εi
as the error term and β1, . . . , βk as the parameters. Write this more concisely in matrix notation:
y = Xβ + ε,
where y and ε are n × 1 vectors, X is a n × k matrix, and β is a k × 1
vector.
Given data y and X, how can we analyze the relationship between the dependent variable and its regressors?
Eric Eisenstat (School of Economics) ECON3350/7350 Week 1
7 / 16
Assumptions of the Classical Regression
To make this setup useful in practice, we need some assumptions. CR has four basic assumptions:
1 rank(X) = k (there are no linearly dependent regressors),
2 X is nonstochastic (the regressors are fixed in random sampling),
3 E(ε) = 0 (zero-mean errors),
4 Var(ε) = σ2In (errors are uncorrelated and have the same variance).
Eric Eisenstat (School of Economics) ECON3350/7350 Week 1 8 / 16
Estimation
A sensible approach to estimate β is to minimize the sum of squared errors:
β = argmin(y − Xβ)′(y − Xβ). This leads to the ordinary least squares (OLS) estimator:
β = (X′X)−1X′y.
As long Assumption 1 (full rank) is satisfied, OLS can be computed.
STATA and a lot of other computer packages can compute OLS even with very large n and k almost instantaneously.
Given assumptions 2-4, OLS is the Best Linear Unbiased Estimator.
Eric Eisenstat (School of Economics) ECON3350/7350 Week 1 9 / 16
Mean Square Error
Is BLUE necessarily the best we can do?
When choosing an estimator, a typical goal is to minimize:
MSE = tr E (β − β)(β − β)′
′
=trVar(β)+tr E(β)−β E(β)−β .
For all unbiased estimators, E(β) = β and MSE is equivalent to the
variance. But we can get lower MSE by trading off variance for bias. Do we really need unbiased estimates?
Eric Eisenstat (School of Economics) ECON3350/7350 Week 1
10 / 16
Unbiased vs. Consistent
An unbiased estimator gets it right “on average” in small samples.
Instead, we might seek a consistent estimator: one that converges (in p
probability) to the true value with the sample size: β −→ β.
An unbiased estimator is always consistent, but a consistent estimator can
be biased in “small” samples.
Hence, we can achieve lower MSE, but still ensure that the estimator “gets it right” if the sample is “large enough”.
The consistent estimator that guarantees lowest MSE is the maximum likelihood estimator (MLE).
turns out to be equivalent to OLS under certain conditions
Eric Eisenstat (School of Economics) ECON3350/7350 Week 1 11 / 16
Regression Example
Suppose we want to analyze the relationship between wages, years of experience, and years of schooling in the population.
We specify the model:
wagei = β1 + β2YOEi + β3YOSi + ε.
We assume that (wagei, YOEi, YOSi) have a tri-variate distribution. The regression can help us draw inference about the conditional expectation
E(wagei | YOEi, YOSi) = β1 + β2YOEi + β3YOSi.
Eric Eisenstat (School of Economics) ECON3350/7350 Week 1 12 / 16
Causal inference from the CR
What about the question: how does an additional year of schooling affect expected earnings?
Have to be very careful about causal inference!
The derivative ∂E(wagei | YOEi, YOSi)/∂YOSi can be ascertained as long as YOSi does not affect the other observed regressors (i.e. YOEi) nor any unobserved regressors (jointly hidden in the error term).
In general, CR makes no claims about the direction of causality. If the assumption are fulfilled, then β3 is consistently estimated, but if we switch wagei with YOSi in the regression, we will still get consistent estimates!
Eric Eisenstat (School of Economics) ECON3350/7350 Week 1 13 / 16
Inference from the CR with Hypothesis Tests
In the classical context, inference is obtained through hypothesis tests.
By introducing an additional assumption regarding the distribution of error
term, for example ε ∼ N (0, σ2In), we obtain a sampling distribution for β. Use the sampling distribution to test relationships, e.g.
H0 : β3 = 0, H1 : β3 ̸= 0 – rejecting H0 provides statistical evidence that wagei is correlated with YOSi.
H0 :β2 =β3 =0,H1 :β2 ̸=0orβ3 ̸=0–rejectingH1 provides evidence that wagei is correlated with at least one of the regressors.
Keep in mind that:
A null hypothesis is never accepted (fail to reject means not enough
statistical evidence to reject H0 at the given significance level). For a fixed significance level, almost any H0 can be rejected with a
large enough sample size.
Eric Eisenstat (School of Economics) ECON3350/7350 Week 1 14 / 16
Choice of Regressors
The right choice of regressors is critical, but extremely difficult.
This is an important issue in applied macro–we will return to it.
Some guidelines to consider:
Never discard regressors due to lack of significance.
Do not rely on goodness-of-fit measures such as R2, which never decreases as more regressors are added.
Omitting a regressor might violate Assumption 3, but might also uphold Assumption 3!
Good approach: try several specifications; use information criteria to compare them; report and interpret results across all specifications.
Eric Eisenstat (School of Economics) ECON3350/7350 Week 1 15 / 16
Application to Macro and Finance
In macroeconomics and finance, most data is time-series and variables are endogenous; hence, Assumptions 2-4 rarely hold.
Use CR as a starting point, but develop more suitable methods. Next week, start discussing univariate time-series.
Eric Eisenstat (School of Economics) ECON3350/7350 Week 1 16 / 16