ECONOMETRICS I ECON GR5411
Lecture 17 – Feasible GLS and Instrumental Variables
by
Seyhan Erden Columbia University MA in Economics
Outline for today:
1. Finishing the discussion of GLS estimator 2. Feasible GLS
3. Asymptotic properties of Feasible GLS
4. InstrumentalVariables 1. Why do we need them?
2. Examples 3. Estimation
11/11/20 Lecture 17 GR5411 by Seyhan Erden 2
Feasible GLS:
When Ω is known. It is easy to estimate the
GLS.
𝛽” = (𝑋*Ω+,𝑋)+,𝑋*Ω+,𝑦 $%&
In practice, 𝜎0 and Ω are unknown.
The feasible GLS estimator replaces 𝜎0Ω
by consistent estimates, 𝜎10 and Ω2 satisfying two conditions such that imply that the
” ”
feasible GLS, 𝛽3$%& (or 𝛽), is ”
asymptotically equivalent to the GLS, 𝛽$%&. 11/11/20 Lecture 17 GR5411 by Seyhan Erden 3
Feasible GLS:
Feasible GLS estimator is
” *2+, +, *2+, 𝛽=(𝑋Ω 𝑋) 𝑋Ω 𝑦
Conditions that imply that 𝛽” is 3$%&
asymptotically equivalent to 𝛽” $%&
are
𝑝𝑙𝑖𝑚 𝑛1 𝑋*Ω2+,𝑋 − 𝑛1 𝑋*Ω+,𝑋
= 0
𝑝 𝑙 𝑖 𝑚 1 𝑋 * Ω2 + , 𝜀 − 1 𝑋 * Ω + , 𝜀 = 0 𝑛𝑛
11/11/20 Lecture 17 GR5411 by Seyhan Erden 4
Feasible GLS:
𝑝𝑙𝑖𝑚 𝑛1 𝑋*Ω2+,𝑋 − 𝑛1 𝑋*Ω+,𝑋 = 0
This one states that if the weighted sum of squares matrix based on the true Ω converges to a positive definite matrix, then the weighted sum of square matrix based on Ω2 converges to the same positive definite matrix.
11/11/20 Lecture 17 GR5411 by Seyhan Erden 5
Feasible GLS:
𝑝 𝑙 𝑖 𝑚 1 𝑋 * Ω2 + , 𝜀 − 1 𝑋 * Ω + , 𝜀 = 0 𝑛𝑛
If the transformed regressors 𝑋∗ are well behaved, then the right-hand-side sum will have a limiting normal distribution.
These conditions must be verified case-by-case basis.
If there is no restriction on Ω, there are too many parameters to estimate, typically there will be some restrictions. The typical problem involves a small set of parameters 𝜶 such that Ω = Ω(𝜶)
11/11/20 Lecture 17 GR5411 by Seyhan Erden 6
Feasible GLS:
For example, a typical Ω in time-series setting is 𝜀? = 𝜌𝜀?+, + 𝑣?
Then
1 1−𝜌0
Ω(𝜌)=𝜎0 𝜌⋮ ⋱
𝜌D+, …
1−𝜌0
𝜌… 𝜌D+,
𝜌D+0
⋮ 1
which involves only one additional unknown parameter, 𝜌. Finite sample properties of FGLS is unknown
11/11/20 Lecture 17 GR5411 by Seyhan Erden 7
Feasible GLS:
If we assume that the two conditions above are met, then the FGLS estimator based on 𝜶G has the same asymptotic properties as the GLS estimator. This result is very helpful.
Theorem: Efficiency of FGLS Estimator
An asymptotically efficient FGLS estimator does not require that we have an efficient estimator of 𝜶; only a consistent one is required to achieve full efficiency for the FGLS estimator.
11/11/20 Lecture 17 GR5411 by Seyhan Erden 8
Feasible GLS:
In two studies, it is found that over a broad range of parameters, FGLS is more efficient than OLS, but if the departure from classical assumptions is not too severe, then OLS may be more efficient than FGLS in small samples.
11/11/20 Lecture 17 GR5411 by Seyhan Erden 9
the consistency of GLS or FGLS require:
Last term is
𝛽” $%&
= (𝑋*Ω+,𝑋)+,𝑋*Ω+,𝑦
* +, +, * +, =𝛽+(𝑋Ω 𝑋) 𝑋Ω 𝜀
1* +,1*
𝑛 𝑋 Ω𝑋 𝑛 𝑋 ΩƐ
We need the first term in parenthesis to approach in probability to a finite number and the second term to zero but the second term may have non-zero off-diagonal elements
𝑛1𝑋*ΩƐ=𝑛1JJ𝑥K Ω+, KL𝜀L KL
for 𝑖 ≠ 𝑗 so it is not enough to assume 𝐸 𝜀K|𝑥K = 0 we must also assume 𝐸 𝜀K|𝑥L = 0 for 𝑖 and 𝑗 when
Ω+, KL is 10
non-zero.
11/11/20 Lecture 17 GR5411 by Seyhan Erden
About Exogeneity assumption of GLS:
Exogeneity assumption under OLS is
𝐸𝜀K|𝑥K =0
However the exogeneity assumption for GLS must be
𝐸 𝜀K|𝑥,,𝑥0,…,𝑥D = 𝐸 𝜀K|𝑋 = 0
Because if off-diagonal elements of Ω matrix is not zero,
then the last expression
𝑛1𝑋*ΩƐ=𝑛1JJ𝑥K Ω+, KL𝜀L KL
for 𝑖 ≠ 𝑗 so it is not enough to assume 𝐸 𝜀K|𝑥K = 0 we must also assume 𝐸 𝜀K|𝑥L = 0 for 𝑖 and 𝑗 when
Ω+, KL is non-zero.
11/11/20 Lecture 17 GR5411 by Seyhan Erden 11
The Instrumental Variables
We will begin with IV regression model and the two stage least squares (TSLS) estimator and its asymptotic distribution in the general case of heteroskedasticity, all in matrix form.
Then, we show that under the special case of homoskedasticity, the TSLS estimator is asymptotically efficient among the class of IV estimators in which the instruments are linear combinations of exogenous variables.
Then we discuss J-statistic for overidentifying restrictions.
We conclude with a discussion of efficient IV estimation and the test of over identifying restrictions when the errors are heteroskedastic – a situation in which the efficient IV estimator is known as the efficient generalized method of moments (GMM) estimator.
11/11/20 Lecture 17 GR5411 by Seyhan Erden 12
There is endogeneity in the linear model 𝑦 K = 𝑥 K* 𝛽 + 𝜀 K
if 𝛽 is the parameter of interest and 𝐸 𝑥K𝜀K ≠ 0
This is a core problem in econometrics and largely differentiates the field of econometrics from the field of statistics.
We say 𝑋 is endogenous for 𝛽. 𝛽”→T 𝐸𝑥K𝑥K* +,𝐸𝑥K𝑦K
𝛽”→T𝛽+𝐸𝑥K𝑥K* +,𝐸𝑥K𝜀K ≠𝛽
The inconsistency of least squares is referred to as endogeneity bias (although here we show inconsistency not the bias). Endogeneity requires
alternative estimation to least squares.
11/11/20 Lecture 17 GR5411 by Seyhan Erden 13
Examples to Endogeneity
The concept of endogeneity may be easiest to understand by example.
Example 1: Measurement error in the regressor. Suppose 𝑧K and 𝑦K are joint random variables,
𝐸 𝑦K|𝑧K = 𝑧K*𝛽 is linear (note that 𝑧K is k×1)
Now assume that 𝑧K is not observed, instead we observe 𝑥K = 𝑧K + 𝑢K where 𝑢K is 𝑘×1 measurement error, independent of 𝜀K and 𝑧K. This is an example of a latent variable model we
have seen before.
11/11/20 Lecture 17 GR5411 by Seyhan Erden 14
Examples to Endogeneity:
Themodel𝑥K =𝑧K +𝑢K with𝑧K and𝑢K independent and 𝐸 𝑢K = 0 is known as classical measurement error. In this case 𝑥K is noisy but unbiased of 𝑍.
We can express 𝑌 as a function of the observed variable 𝑋:
𝑦K =𝑧K*𝛽+𝜀K = 𝑥K −𝑢 *𝛽+𝜀K =𝑥K*𝛽+𝑣K where 𝑣K = 𝜀K − 𝑢K*𝛽
11/11/20 Lecture 17 GR5411 by Seyhan Erden 15
Examples to Endogeneity:
Note that for this model *
𝑦K =𝑥K𝛽+𝑣K
The error is not a projection error, * 𝐸𝑥K𝑣K =𝐸𝑧K+𝑢K 𝜀K−𝑢K𝛽
= − 𝐸 𝑢 K 𝑢 K* 𝛽 ≠0
if𝛽≠0and𝐸𝑢K𝑢K* ≠0
Hence, least squares estimator will be inconsistent
𝛽”→T𝛽+𝐸𝑥K𝑥K* +,𝐸𝑥K𝑣K ≠𝛽
11/11/20 Lecture 17 GR5411 by Seyhan Erden 16
Examples to Endogeneity with Hansen’s new notation (Oct 2020 ed.):
I want to mention this new notation, because it may become more common:
The model 𝑋 = 𝑍 + 𝑢 with 𝑍 and 𝑢 independent and 𝐸 𝑢 = 0 is known as classical measurement error. In this case 𝑋 is noisy but unbiased of 𝑍.
We can express 𝑌 as a function of the observed variable 𝑋:
𝑦=𝑋*𝛽+𝜀= 𝑋−𝑢 *𝛽+𝜀=𝑋*𝛽+𝑣
where 𝑣 = 𝜀 − 𝑢′𝛽
11/11/20 Lecture 17 GR5411 by Seyhan Erden 17
Back to old notation: Examples to Endogeneity:
The inconsistency of least squares
𝛽 ” →T 𝛽 + 𝐸 𝑥 K 𝑥 K * + , 𝐸 𝑥 K 𝑣 K ≠ 𝛽
This is called measurement error bias or attenuation bias.
With the new notation:
𝛽” →T 𝛽 + 𝐸 𝑋 𝑋 ′ + , 𝐸 𝑋 𝑣 ≠ 𝛽
11/11/20 Lecture 17 GR5411 by Seyhan Erden 18