Econometrics (M.Sc.) 1 Exercises (with Solutions) Chapter 4
1. Problem
Consider the following multiple linear regression model:
Yi =β1 +β2Xi2 +β3Xi3 +εi i=1,…,n, where n is a small sample size (e.g. n = 15).
(a) How can one test the hypothesis that β3 = 1 against a two-sided alternative. State H0, HA, the test-statistic, its distribution, and explain how the test decision is conduced.
(b) Consider the null hypothesis β2 + β3 = 0 against a two-sided alternative. Use mathe- matical derivations to show that in this case an F -test simplifies to a t-test.
(c) How can one test the hypothesis that β2 = β3 = 0. State H0, HA, the test-statistic, its distribution, and explain how the test decision is conduced.
Solution
(a) The null hypothesis H0 : β3 = 1 can be tested versus the two-sided alternative HA : β3 ̸= 1 by means of a t-test. The test statistic is
where
βˆ 3 − 1 t= ,
ˆ SE β3|X
33
−1 ˆ2′
SE β3|X = sUB (XX)
.
Under the null hypothesis and under Assumptions 1-4∗, the test statistic t has a t distribution with (n − K) = (n − 3) degrees of freedom. At the α = 5% significance level, we reject the null if |tobs| > cα/2 (two-tailed test), where tobs is the observed value of the test statistic, and where cα/2 is the (1 − α/2) quantile of the t distribution with (n − 3) degrees of freedom.
(b) The hypothesis H0 : β2 +β3 = 0 can be tested by means of a t-test since it corresponds effectively to one single hypothesis (i.e., q = 1). The null hypothesis is equivalent to H0 :Rβ−r=0withR=(0,1,1)andr=0.Thatis,theF-testsimplifiestoat-testas following:
F = (Rβˆn − r)′[R(s2UB(X′X)−1)R′]−1(Rβˆn − r)/q
F = (Rβˆn − r)′(Rβˆn − r) (Since R(s2UB(X′X)−1)R′ is a scalar and q = 1.)
R ( s 2U B ( X ′ X ) − 1 ) R ′
βˆ + βˆ 2
23
F= −1 −1 −1
⇔
H 0
t= ∼t(n−3)
F =
ˆˆ
s2UB (X′X) + s2UB (X′X) + 2s2UB (X′X)
22 33 23
βˆ + βˆ 2 23
F= ˆ ˆ ˆˆ
Var β2|X +Var β3|X +2·Cov β2,β3|X
βˆ + βˆ 2 23
H0
∼ F(1,n−3)
Var β2+β3|X βˆ 2 + βˆ 3
ˆˆ SE β2+β3|X
Econometrics (M.Sc.) 2 (Remember that for two random variables Y and Z we have Var(Y + Z) = Var(Y ) +
Var(Z)+2Cov(Y,Z))
(c) The joint null hypothesis that β2 = β3 = 0 can be tested by means of an F -test. The nullhypothesisisH0 :Rβ−r=0withthefollowing(q×K)=(2×3)matrixRand q = 2 vector r :
0 1 0 0 R= 0 0 1 and r= 0 .
The alternative hypothesis is HA : Rβ − r ̸= 0. The test statistic is given by F = (Rβˆn − r)′[R(s2UB(X′X)−1)R′]−1(Rβˆn − r)/q
We compare the observed test statistic, Fobs, with the critical value cα from an F distribution with q = 2 (the number of restrictions) and n−3 degrees of freedom, where cα is the (1 − α) quantile of the F distribution with (q,n − K) = (2,n − 3) degrees of freedom. If Fobs > cα, we reject the null hypothesis.
2. Problem
Correct or false? Justify your answers. (You do not need to show orthogonality between the regressors and the residuals.)
a) A regression of the OLS residuals on the regressors included in the model by con- struction yields an R2 of zero.
b) If an estimate βˆk is significantly different from zero at the 10% level, it is also signifi- cantly different from zero at the 5% level.
Solution
a) Correct. This follows from the orthogonality of Xk and ˆε for all k = 1, . . . , K, i.e. X′ˆε = 0, where 0 is here a K × 1 column-vector. This result was derived under the OLS estimator and used for the method of moments estimator. In a regression of ˆε on X, the OLS estimator would be γˆ = (X′X)−1X′ˆε which is a vector of zeros (γˆ = 0) since X′ˆε = 0. Let denote the fitted residuals by e = Xγˆ and note that e = 0 since γˆ = 0. Consequently, we also have that e ̄ = n−1 ni=1 ei = 0. This now allows us to show that R2 = 0 is this case (remember ̄ˆε = n−1 ni=1 ˆεi = 0 since the intercept (1,…,1)′ andˆεareorthogonal):
2 ni=1 (ei − e ̄)2 0 R=n ̄2=n ˆε2=0
i=1 ˆεi−ˆε i=1 i
b) False. Actually, it works the other way around. If an estimate is significantly different from zero at the 5% level, it is also significantly different from zero at the 10% level. This is most easily explained on the basis of the p -value. If e.g. the p -value belonging to the observed value of the t test statistic, tobs, is smaller than 0.05 (5%), the null hypothesis H0 : βk = 0 can be rejected and we say that the estimate βˆk is significantly different from zero at the 5% level. Clearly, if p is smaller than 0.05 it is also smaller than 0.10 (10%) but not vice versa.
3. Problem
An alternative, equivalent representation of the F -test statistic is the following:
ni=1 ˆε2iR − ni=1 ˆε2iU /q (SSR − SSU ) /q F = ni=1ˆε2iU/(n−K) = SSU/(n−K) ,
Econometrics (M.Sc.) 3
where ˆεiU are the residuals from the unrestricted (i.e., the usual) regression of Y on X, and where ˆεiR are the residuals from the restricted ordinary least squares regression which minimizes the following restricted version of the OLS-objective function
minSn(β ̃) = (Y −Xβ ̃)′(Y −Xβ ̃) such that Rβ ̃−r = 0 β ̃
where the restriction is just the null hypothesis.
The standard F-test. The standard F-test for a linear regression tests the hypothesis that all coefficients except the intercept are equal to zero. In this case, ˆεiR are simply the residuals from regressing Y on only the intercept. In this standard case we have,
ni=1(Yi −Y ̄)2 −ni=1ˆε2iU/(K−1) F1 = ni=1ˆε2iU/(n−K)
sincehereˆε2iR =(Yi −Y ̄)2. Show that F1 is equal to F2 with
F2= RU2/(K−1) , (1−RU2 )/(n−K)
where RU2 denotes the coefficient of determination of the unrestricted regression model. Solution
ni=1(Yi −Y ̄)2 −ni=1ˆε2iU ni=1(Yi−Y ̄)2
By the definition of the RU2 we have that 2 ni=1ˆε2iU
RU =1−ni=1(Yi−Y ̄)2 = 2 ni=1 ˆε2iU
1−R =ni=1(Yi−Y ̄)2 Inserting RU2 and 1 − RU2 into F2 yields:
F2 = RU2 /(K − 1) (1−RU2 )/(n−K)
n ( Y i − Y ̄ ) 2 − n
i=1 i=1 iU
ˆε 2
n ̄2 /(K−1)
= n ˆε2 i=1 iU
i=1(Yi−Y )
=
n ̄2 /(n−k) i=1(Yi−Y )
ni=1(Yi −Y ̄)2 −ni=1ˆε2iU/(K−1) ni=1 ˆε2iU /(n − k)
= F1
4. Problem
The Boston housing data set (contained in the R package MASS) contains observations on housing values in suburbs of Boston. Let’s consider the following regression model
medvi = β1 + β2ptratioi + β3lstati + β4noxi + β5crimi + εi
where i = 1, . . . , n indexes the suburbs. Check ?Boston in R to get an overview about the variables. The following R code computes the regression estimates:
Econometrics (M.Sc.) 4
library(“lmtest”) # for coeftest()
library(“MASS”) # for Boston housing data
data(“Boston”) # ?Boston; names(Boston)
# focusing here on a small sample case (n=20):
Boston_small <- Boston[1:20,]
lm_obj <- lm(medv ~ ptratio + lstat + nox + crim, data = Boston_small)
round(coeftest(lm_obj), 2)
##
## t test of coefficients:
##
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 112.59
## ptratio -0.92
## lstat -0.47
## nox -134.72
## crim 4.99
## —
39.86 2.82
1.00 -0.92
0.18 -2.57
53.73 -2.51
8.65 0.58
0.01 **
0.37
0.02 *
0.02 *
0.57
## Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ‘ 1
(a) UseRtotestH0 :β3 =0versusHA :β3 <0bymeansofant-test.Whatisthecorrect
p-value and what is the test decision when using a significance level of α = 0.015?
(b) UseRtotestH0 :β3 =β4 =0versusHA :β3 ̸=0and/orβ4 ̸=0bymeansofan
F -test. What is the marginal significance value in this case?
(c) What is the maximal probability of a type I error if you test the null hypothesis in (b) by means of two separate t-tests instead of one F -test? How does this compare to the probability of a type I error for the F test in (b).
Solution
(a) The standard regression output reports two-sided t-tests. The observed value of the t-test statistic for βˆ3 is tobs = −2.57 with a two-sided p-value ptwo-sided = 0.02, where
ptwo-sided = 2 min{PH0 (t ≤ tobs), PH0 (t ≥ tobs)}.
pone-sided, lower pone-sided, upper
The correct p-value for HA : β3 < 0, however, is pone-sided, lower. Since tobs = −2.57 is
negative we know that PH0 (t ≤ tobs) < PH0 (t ≥ tobs). Therefore, ptwo-sided = 2pone-sided, lower
which allows us to compute the correct pone-sided, lower by pone-sided, lower = ptwo-sided/2
= 0.02/2 = 0.01.
That is, we can reject the null-hypothesis H0 : β3 = 0 against the alternative HA : β3 <
0 at the significance level of α = 0.015.
(b) AtestofH0 :β3 =β4 =0versusHA :β3 ̸=0and/orβ4 ̸=0bymeansofanF-test can be conducted as following:
Econometrics (M.Sc.) 5 library("car")
linearHypothesis(lm_obj, c("lstat=0", "nox=0"))
## Linear hypothesis test
##
## Hypothesis:
## lstat = 0
## nox = 0
##
## Model 1: restricted model
## Model 2: medv ~ ptratio + lstat + nox + crim
##
##
## 1
## 2
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
So, the marginal significance value (p-value) in this case is 0.01177. That is, we can reject the null hypothesis at any significance level α > 0.01177.
(c)WhenwetestH0 :β3 =β4 =0versusHA :β3 ̸=0and/orβ4 ̸=0bymeansof two separate t-tests, we may conduct a type I error in either of the two test-decisions. Therefore, the joint probability of a type I error is
PH0 (|t(3)| > cα/2 ∪ |t(4)| > cα/2),
where t(3) and t(4) denote here the t-tests based on βˆ3 and βˆ4 respectively. Using that P(A∪B) = P(A)+P(B)−P(A∩B) (see Script, Chapter 1.1.2) we can derive the fol- lowing upper threshold for this joint probability of type I errors (assuming Assumptions 1-4∗ hold):
PH0 (|t(3)| > cα/2 ∪ |t(4)| > cα/2) = PH0 (|t(3)| > cα/2) + PH0 (|t(4)| > cα/2) − PH0 (|t(3)| > cα/2 ∩ |t(4)| > cα/2)
≤ PH0 (|t(3)| > cα/2) + PH0 (|t(4)| > cα/2) = 2α
So, the maximal probability of a type I error if you test the null hypothesis in (b) by means of two separate t-tests is two times the significance level α of the two separate t-tests. That is, in order to do inference at a chosen α-level, we need to conduct each of the two t-tests at an α/2 significance level. (This is then called a Bonferroni- adjustment.) Such an adjustment makes it generally harder to detect a violation of the null-hypothesis then when using the F test in (b); particularly, in the case of more than two t-tests.
Res.Df RSS Df Sum of Sq F Pr(>F)
17 344.27
15 190.40 2 153.88 6.0614 0.01177 *