Problem 1: 25 Marks
Consider the following model for yi: xi1 β1
y i = β 0 + X i′ β + ε i .
Here Xi = . and β = . . You may assume that yi and all elements of Xi have mean 0.
(b)
βˆRIDGE =(X′X+λIK)−1X′Y
unbiased estimator of β. Show that βˆRIDGE is a biased estimator of β if λ ̸= 0. (12.5 marks)
1
X2
. .
xi2 β2 . . xik βk
The objective function for RIDGE is given by:
NK βˆRIDGE=argmin(yi−Xi′β)2 subjectto βk2≤c
for some c > 0. Alternatively:
β
i=1
k=1
βˆRIDGE = arg min (yi − Xi′β)2 + λ βk2.
β
i=1 k=1
(a) Derive βˆRIDGE in terms of Xi, yi, and λ.1 (12.5 marks) Solution: We may rewrite the objective function as
βˆRIDGE =argmin(y−Xβ)′(y−Xβ)+λβ′β β
Differentiating with respect to β gives:
X′Y =(X′X+λIk)β.
NK
Solving for β gives:
Suppose the assumptions on εi and Xi are such that the OLS estimator βˆOLS provides an
Solution: Define A = X′X
βˆRIDGE = (X′X + λI )−1X′Y
Therefore, if λ ̸= 0:
X 1′
′
E[βˆRIDGE] = E[(I λk
λK
= (A + λIK )−1A(A−1X′Y )
= (A[Ik + λA−1])−1A(A−1X′Y )
= (Ik + λA−1)−1A−1A((X′X)−1X′Y ) = (Ik + λA−1)−1βˆols
+ λA−1])−1βˆols] ̸= β FeelfreetouseX= . andY =.ifyouprefermatrixnotation.
y 1 y2
Xk′ yk
©Imperial College London 2018/2019
Author: CJH
Problem 2: 25 Marks
Relative to the United Kingdom, the United States has borrower friendly laws surrounding residen- tial mortgage default. Many US states are Non-Recourse—that is, if borrowers stop making the mortgage payments, lenders cannot hold them responsible beyond seizing the home itself. On the other hand, the United Kingdom has Full-Recourse: lenders may seize cars, investments, garnish wages, et cetera. Many believe that the relative leniency of laws in the United States is responsible for higher rates of mortgage default.
For the sake of simplicity, assume laws may take only two forms: Non-Recourse (in the United States) or Full-Recourse (in the United Kingdom). Imagine we are interested in the causal (treat- ment) effect of Non-Recourse laws on mortgage default.
(a) Denote mortgage default for a borrower i by Di. In potential outcomes notation, write the average treatment effect of Non-Recourse laws on default. (5 marks)
Solution: Define Di1 to be the potential outcome for borrower i in the presence of Non- Recourse laws. Define Di0 to be the potential outcome for borrower i under Full-Recourse laws. The average treatment effect is defined to be:
ATE = E[Di1 − Di0]
(b) Suppose we compare the average default rates in the United States to the average default rates
in the United Kingdom. Write this comparison in potential outcomes notation. (5 marks) Solution:
E[Di1|Borrower i in US] − [Di0|Borrower i in UK]
(c) Why does the expression in part (a) differ from that in part (b)? Please provide an explanation that is not simply mathematical, but that provides some intuition. Would you expect the answer in (b) to be higher or lower than that in (a)? Why? (15 marks)
Solution: There are many ways to describe why the expression in (a) and (b) might be different. One way is to break the above into two components:
E[Di1|Borrower i in US] − [Di0|Borrower i in UK] = E[Di1|Borrower i in US] − E[Di0|Borrower i in US]
Effect of Non-Recourse in US
+ E[Di0|Borrower i in US] − [Di0|Borrower i in UK]
Difference in Default in US vs. UK under Full-Recourse ̸= E[Di1 − Di0]
The first term is often referred to as the treatment effect on the treated, and captures the fact that recourse laws might impact borrowers in the US differently than the UK (perhaps because of other regulatory differences or personality types). The second term is often referred to as the selection effect, and captures the fact that borrowers in different countries might have differences in default behavior, even in the absence of any difference in bankruptcy laws.
In general, cogent arguments that (a) is higher or lower than (b) can be made. The important part is to directly tie it to the framing. One example is that borrowers in the US may be less concerned with the social stigma surrounding default than those in the UK. This might be evidence for for the existence of a selection effect:
E[Di0|Borrower i in US] − [Di0|Borrower i in UK] > 0 which would cause (b) to be higher than (a).
Author: CJH
©Imperial College London 2018/2019
Problem 3: 15 Marks
Suppose we are interested in estimating the coefficients β0, β1, and β2 in the following linear model: yi∗ =β0 +β1x1i +β2x2i +vi.
While we observe xi, we are unable to observe yi∗ entirely. Instead, we see yi, where yi is given by:
k if yi∗ ≤ 0 yi= yi∗ifyi∗>0
for some constant k. Let vi ∼ N(0,1) be a standard normal random variable with probability density function f(z|x1i,x2i) = φ(z) and cumulative distribution function F(z|x1i,x2i) = Φ(z).
(a) What is the probability distribution function of yi given x1i, x2i and the parameters β0, β1, β2: g(yi|x1i,x2i;β0,β1,β2)? (15 marks)
Solution:
g(yi|x1i, x2i; β0, β1, β2, ) = 1{yi = k} Φ(−β0 − β1x1i − β2x2i)
+1{yi >0} φ(yi −β0 −β1x1i −β2x2i)
Author: CJH
©Imperial College London 2018/2019
Problem 4: 10 Marks
Suppose we see 5 observations of yi, Di as shown in the table below:
Consider the following linear mode:
yi Di 30 81
11 1 00 81
yi = δ0 + δ1Di + vi.
(a) SupposeweestimatethismodelonthedataaboveviaOLS.PleaseexplicitlyfindδˆOLS andδˆOLS. 01
(10 marks)
Solution: δˆOLS = 1.5, δˆOLS = 7.5
01
Author: CJH
©Imperial College London 2018/2019
Problem 5: 10 Marks
Suppose we see excess returns xit on 2 assets over T time periods (t = 1, 2, · · · , T ). We may write these together as a vector at time t:
x1t xt=x .
2t
You may assume that each excess return has mean 0 and that the empirical covariance matrix is
given by .
1 0 Cov(xt)=Σx = 0 3
(a) Define pt the set of principle components variables of xt (8 marks)
Solution:
x2t pt= x1t
ˆˆ
(b) What fraction of the total variance of xt can be explained by the first principle component? (2 marks)
Solution: 3 4
Author: CJH
©Imperial College London 2018/2019
Problem 6: 15 Marks
(a) Discuss the bias-variance tradeoff in prediction exercises. Why does this tradeoff arise? Are there problems in which we might we prefer a biased estimator to an unbiased one? How might this relate to the complexity of a prediction model? (15 marks)
Solution: An appropriate solution will
• Describe an out-of-sample prediction problem
• Mention the mean-squared-error or some other measure of fit
• Describe techniques discussed in lecture (e.g. LASSO) that are biased but may improve out of sample fit.
• Discuss the potential for variance to increase with model complexity.
An excellent solution will explicitly decompose the mean squared error in terms of bias and variance.
Author: CJH
©Imperial College London 2018/2019