ECONOMETRICS I ECON GR5411
Lecture 12 –Hypothesis Testing, Restricted Least Squares
by
Seyhan Erden Columbia University MA in Economics
Hypothesis Testing:
Start with the regression model
𝑦 = 𝑋𝛽 + 𝜀 Null hypothesis: 𝐻(
ØNull and Alternative hypotheses are exclusive and exhaustive, within the set of admissible hypotheses, there is no 3rd possibility either one or the other is true, not both.
ØReject 𝐻( when the data are inconsistent with a hypothesis with a reasonable degree of certainty.
ØDo not reject 𝐻( when the data appear to be
consistent with the null hypothesis (not the same
as “accepting” 𝐻()
Lecture 12 GR5411 by Seyhan Erden 2
Nested vs Non-nested Models:
If the model in 𝐻( is a special case of the model in 𝐻), the two models are said to be nested. The smaller model is the restricted model and the larger one is unrestricted model, such as;
Unrestricted model:
𝑦 = 𝛽( +𝛽)𝑋) +𝛽*𝑋* +𝛽+𝑋+ +𝜀
Restricted model:
𝑦 = 𝛽( +𝛽)𝑋) +𝛽*𝑋* +𝜀
𝐻(: 𝛽+=0
𝐻): 𝛽+≠0
Lecture 12 GR5411 by Seyhan Erden 3
Nested vs Non-nested Models:
If the model in 𝐻( is different from the model in 𝐻), the two models are said to be non-nested.
Such as
𝐻(: 𝑦=𝛽( +𝛽)𝑋) +𝛽+𝑋+ +𝜀 𝐻): 𝑦=𝛽( +𝛽)𝑋) +𝛽*𝑋* +ε
Lecture 12 GR5411 by Seyhan Erden 4
Neyman – Pearson Approach:
ØBased on the data , construct the rejection region, such as
ØForm a test statistic based on the estimated coefficients. ØCheck if the test statistic is in the rejection region using
a rule, if so reject 𝐻(
ØThe estimated coefficients are random and the decision rule can be wrong.
Lecture 12 GR5411 by Seyhan Erden 5
Significance Testing:
ØThe estimated coefficients are random and the decision rule can be wrong.
STATISTICAL DECISION
TRUE STATE OF NULL HYPOTHESIS
𝐻( is true
𝐻( is false
𝑅𝑒𝑗𝑒𝑐𝑡 𝐻(
Type I error
Correct
𝐷𝑜 𝑛𝑜𝑡 𝑟𝑒𝑗𝑒𝑐𝑡 𝐻(
Correct
Type II error
Lecture 12 GR5411 by Seyhan Erden 6
Significance Testing:
Ø Type I error: Rejecting a true null hypothesis, (reject 𝐻( when it is actually true but the sample wrongly rejects it)
ØFalse positive: telling a man he is pregnant.
ØType II error: Not rejecting a false null hypothesis (not rejecting 𝐻( when it is actually false)
ØFalse negative: telling a pregnant woman that she is not pregnant.
Lecture 12 GR5411 by Seyhan Erden 7
Size and Power of the Test:
Size of the test: 𝑃𝑟𝑜𝑏(𝑇𝑦𝑝𝑒 𝐼 𝑒𝑟𝑟𝑜𝑟) Probability of rejecting a true 𝐻(
Power of the test: 1 − 𝑃𝑟𝑜𝑏(𝑇𝑦𝑝𝑒 𝐼𝐼 𝑒𝑟𝑟𝑜𝑟)
Probability that it will correctly reject a false null hypothesis against a given alternative.
ØA test is consistent if its power goes to 1 as the sample size increases
ØNeyman-Pearson approach: user sets the size of the test
Lecture 12 GR5411 by Seyhan Erden 8
General Linear Hypothesis:
Consider the linear model
𝑦 = 𝑋𝛽 + 𝜀
Restrictions are
𝑅′𝛽 = 𝑞
Where 𝑅 is 𝑘×𝑗 matrix to select necessary 𝛽s,
𝛽 is the usual 𝑘×1 vector of coefficients and 𝑞 is the 𝑗×1 vector of restrictions.
Each row of 𝑅 is the coefficients in one of the restrictions, 𝑗 must be less than 𝑘 (why?) the rows of 𝑅 must be linearly independent.
There are 𝑗 restrictions and 𝑘 − 𝑗 free parameters.
Lecture 12 GR5411 by Seyhan Erden 9
General Linear Hypothesis:
Typically 𝑅 will have one or a few rows and numerous zeros on each row. The hypothesis implied by the restrictions is written
𝐻(:𝑅′𝛽 = 𝑞, 𝐻):𝑅′𝛽 ≠ 𝑞
Examples:
ØOne of the coefficients is zero, 𝛽P = 0
𝑅′= 00… 10… 0; 𝑞=0
ØTwo of the coefficients are equal, 𝛽P = 𝛽S 𝑅′=001…0−1…0; 𝑞=0
Lecture 12 GR5411 by Seyhan Erden 10
General Linear Hypothesis:
Ask students:
Let 𝑘 = 3, and you want to test 𝐻(: 𝛽) + 𝛽* = 1
writedown 𝑅,𝛽,𝑗,𝑞? Answer:
0
𝑅 = 1 , 𝑗 = 1, 𝑞 = 1
1
𝛽(
011 𝛽) =1
𝛽*
Lecture 12 GR5411 by Seyhan Erden 11
General Linear Hypothesis:
Examples (cont’):
ØA set of the coefficients sum to one,
𝛽* + 𝛽+ + 𝛽U = 1
Ø𝑅′= 01110… 0; 𝑞=1
ØA subset of coefficients are all zero, 𝛽) =𝛽* =𝛽+ =0
1000…0 0 𝑅′= 0 1 0 0 … 0 = 𝐼|0; 𝑞= 0
0010…0 0
Lecture 12 GR5411 by Seyhan Erden 12
General Linear Hypothesis:
Examples (cont’):
ØSeveral linear restrictions, 𝛽) + 𝛽* = 1,
𝛽U+𝛽W=0, 𝑎𝑛𝑑𝛽Z+𝛽W=0
0110000 1 𝑅′ = 0 0 0 0 1 0 1 ; 𝑞 = 0 0000011 0
ØAll the coefficients in the model except the
constant term are zero, 𝑅 = 0|𝐼S[) ; 𝑞 = 0 Lecture 12 GR5411 by Seyhan Erden
13
Consider 𝑗 linear restrictions: 𝐻(: 𝑅′𝛽−𝑞=0
𝐻): 𝑅′𝛽−𝑞≠0
Discrepancy vector: 𝑅′𝛽\ − 𝑞 = 𝑚
𝑚 is not likely to be exactly equal to zero.
Questions is whether |𝑚 − 0| (deviation of 𝑚 from zero is caused by sampling variability) is 𝑚 really significantly different from zero?
Since 𝛽\ is normally distributed and since 𝑚 is a linear function of 𝛽\, 𝑚 is also normally
distributed.
Lecture 12 GR5411 by Seyhan Erden 14
Mean vector and covariance matrix of 𝑚:
Mean vector of 𝑚: \ 𝐸𝑚𝑋 =𝐸𝑅′𝛽−𝑞|𝑋
= 𝑅 ′ 𝐸 𝛽\ 𝑋 − 𝑞
= 𝑅′𝛽 − 𝑞 =0
Covariance matrix of 𝑚: \ 𝑣𝑎𝑟𝑚𝑋 =𝑣𝑎𝑟𝑅′𝛽−𝑞|𝑋
= 𝑅′ 𝑣𝑎𝑟(𝛽\|𝑋) 𝑅 = 𝑅′ 𝜎*(𝑋a𝑋)[) 𝑅
Lecture 12 GR5411 by Seyhan Erden 15
A test of 𝐻( on Wald criterion : Conditioned on 𝑋, we write Wald test statistic as 𝑊=𝑚a𝑣𝑎𝑟𝑚𝑋 [)𝑚
= ( 𝑅 ′ 𝛽\ − 𝑞 ) ′ 𝑅 a 𝜎 * 𝑋 a 𝑋 [ ) 𝑅 [ ) ( 𝑅 ′ 𝛽\ − 𝑞 )
This is using Theorem B.11 (Greene) If𝑋~𝑁(𝜇,Σ)then 𝑋−𝜇 a Σ[) 𝑋−𝜇 ~𝜒h*
According to this theorem the Wald criterion above must be distributed with Chi-square also,
( 𝑅 ′ 𝛽\ − 𝑞 ) ′ 𝑅 a 𝜎 * 𝑋 a 𝑋 [ ) 𝑅 [ ) ( 𝑅 ′ 𝛽\ − 𝑞 ) ~ 𝜒 P*
Lecture 12 GR5411 by Seyhan Erden 16
A test of 𝐻( on Wald criterion :
Intuitively as 𝑚 gets larger, remember 𝑚 is the difference between 𝑅𝛽 and 𝑞, this difference becomes different from zero, (that is the worse failure of LS to satisfy the restrictions) the larger is the Chi-square statistic.
Thus a large Chi-square statistic will weigh against 𝐻(
However, Chi-square is not usable because 𝜎* is unknown in
(𝑅a𝛽\ −𝑞)′ 𝑅a 𝜎* 𝑋a𝑋 [) 𝑅 [) (𝑅a𝛽\ −𝑞)~𝜒P* Lecture 12 GR5411 by Seyhan Erden 17
A usable test:
How can we use 𝑠* instead of 𝜎*
Recall the distribution theory: Ratio of two chi- squares divided by their degrees of freedom follows F distribution with numerator’s and denominator’s degrees of freedom. If we have
𝑊 ~ 𝜒 P*
(𝑛 − 𝑘)𝑠* * 𝜎* ~𝜒h[S
in the denominator, then
𝑊/𝑗 ~𝐹P,h[S 𝑠*/𝜎*
in the numerator, and
Lecture 12 GR5411 by Seyhan Erden 18
Must prove that (h[S)lm ~ 𝜒* : nm h[S
Fact: If 𝑍 is normal and 𝑀 is idempotent with size 𝑛 − 𝑘 then 𝑍a𝑀𝑍~ 𝜒*
a h[Sa
𝑛−𝑘 𝜀𝑀𝜀 = 1𝜀 𝑀 1𝜀 =𝑍a𝑀𝑍 𝜎* (𝑛−𝑘) 𝜎 𝜎
𝜀~𝑁 0, 𝜎*𝐼 𝜎𝜀 ~ 𝑁 ( 0 , 𝐼 )
M is idempotent and rank of 𝑀 is 𝑛 − 𝑘, then
(𝑛 − 𝑘)𝑠* * 𝜎* ~𝜒h[S
Lecture 12 GR5411 by Seyhan Erden 19