程序代写CS代考 data mining MAST90083 Computational Statistics & Data MiningNonparametric Regression

MAST90083 Computational Statistics & Data MiningNonparametric Regression

Tutorial & Practical 7: Nonparametric Regression

Question 1

Consider the ordinary nonparametric regression model

yi = f(xi) + �i; 1 ≤ i ≤ n

where yi ∈ R, xi ∈ R, �i ∈ R ∼ N (0, σ2) and are i.i.d. For approximating f we propose to
use the linear spline model.

1. Provide the form of this linear spline model and define the set of linear parameters

2. Derive its matrix form and the associated penalized spline fitting criterion

3. To what this criterion corresponds when considering the coefficients of the truncated
linear functions as random with cov(u) = σ2uI where u is the vector of coefficients of
the truncated linear function.

4. Derive the expression for the penalized least squares estimator of the unknown param-
eters of the model

5. Provide the associated expression for the best fitted values

6. Provide a model form which the estimation of the coefficients of the polynomial func-
tions is straightforwardly obtained

7. Derive the estimator of these parameters and their covariance

Question 2

1. Provide the expression of the Nadaraya-Watson kernel estimator and discuss its singu-
larity condition

2. Is the Nadaraya-Watson estimator with a Gaussian Kernel differentiable?

3. Is the Nadaraya-Watson estimator with the Epanechnikov Kernel differentiable?

Question 3

Let’s consider a set of observations generated according to the model

yi = f(xi) + �i; 1 ≤ i ≤ n

where f : [0, 1]→ R, the pairs (xi, yi) , i = 1, …, n are observed and �i are i.i.d. with E(�i) = 0
and E (�2i ) = σ

2
� .

1

MAST90083 Computational Statistics & Data MiningNonparametric Regression

Assuming f ∈ L2 [0, 1] and an orthonormal basis {ρj}

j=1

of L2 [0, 1] then

f(x) =
∞∑
j=1

θjρj(x),

where

θj =

∫ 1
0

f(x)ρj(x)dx.

The projection estimator for f is given by

f̂nN(x) =
N∑
j=1

θ̂jρj(x),

where

θ̂j =
1

n

n∑
i=1

yiρj(xi).

1. Which parameter plays the role of the smoothing parameter in the projection estimator

2. Show that f̂nN is a linear estimator in y

is

3. Show that E
(
θ̂j

)
= θj + rj

4. Show that E

[(
θ̂j − θj

)2]
=

σ2�
n

+ r2j

5. Provide the expression of the MISE = E
[
‖f̂nN − f‖22

]
6. Discuss the effect of N on the MISE

2