MAST90083 Computational Statistics & Data Mining Bootstrap Methods
Tutorial & Practical 9: Solutions
Question 1
1. Given X = {x1, …, xn}, with µ = E (xi)
θ̂ = θ (F1) =
[∫
x
(
1
n
n∑
i=1
δ (x− xi)
)
dx
]3
=
[
1
n
n∑
i=1
xi
]3
= x̄3
2. To find the bias b1 = E
(
θ̂ − θ0
)
= E
(
θ̂
)
− θ0 we need E
(
θ̂
)
= E (x̄3). Let σ2 =
var (xi) = E (xi − µ)
2
and γ = E (xi − µ)
3
be the third cumulant for xi. The third
cumulant of x̄
E (x̄− µ)3 = E
((
1
n
n∑
i=1
xi
)
− µ
)3
=
E
(
1
n
n∑
i=1
(xi − µ)
)3
=
1
n3
E
(
n∑
i=1
(xi − µ)
)3
=
=
1
n3
n∑
i=1
E (xi − µ)
3
=
nγ
n3
=
γ
n2
.
Similarly E (x̄− µ)2 = var (x̄) = σ2/n. For both the variance and the third cumulant,
the cross terms are eliminated by i.i.d. of the samples.
E (x̄)3 = E (x̄− µ+ µ)3
= E
(
µ3 + 3µ2(x̄− µ) + 3µ(x̄− µ)2 + (x̄− µ)3
)
= µ3 +
3µσ2
n
+
γ
n2
Therefore
b1 = E
(
θ̂
)
− θ0 = E
(
θ̂
)
− µ3 =
3µσ2
n
+
γ
n2
= E [θ(F1)− θ(F0)|F0]
3. The bootstrap estimate b̂1 is given by
b̂1 = E [θ(F2)− θ(F1)|F1] =
3x̄σ̂2
n
+
γ̂
n2
where
1
MAST90083 Computational Statistics & Data Mining Bootstrap Methods
x̄ =
∫
xdF1(x) =
1
n
n∑
i=1
xi σ̂
2 =
∫
(x− x̄)2 dF1(x) =
1
n
n∑
i=1
(xi − x̄)2 and
γ̂ =
∫
(x− x̄)3 dF1(x) =
1
n
n∑
i=1
(xi − x̄)3
4. The bootstrap bias reduced estimate is
θ̂1 = θ (F1)− b̂1 = 2θ (F1)− E [θ(F2)|F1]
= x̄3 −
3x̄σ̂2
n
−
γ̂
n2
5. The bias b2 requires the derivation of the expression of E
(
θ̂1
)
.
Using n = 1 in the expression of E (x̄)3 gives E (x1)
3
= µ3 + 3µσ2 + γ. Therefore for any
j
E
(
xj
n∑
i=1
x2i
)
= E
(
x3j
)
+
n∑
i=1,i 6=j
E
(
xjx
2
i
)
= µ3 + 3µσ2 + γ + (n− 1)µ
(
µ2 + σ2
)
averaging over j gives
E
(
x̄
n∑
i=1
x2i
)
= µ3 + 3µσ2 + γ + (n− 1)µ
(
µ2 + σ2
)
Since
σ̂2 =
1
n
n∑
i=1
x2i − x̄
2
E
(
x̄
n∑
i=1
x2i
)
= E
[
1
n
x̄
n∑
i=1
x2i − x̄
3
]
= E
[
1
n
x̄
n∑
i=1
x2i
]
−
[
x̄3
]
= µσ2 +
1
n
(
γ − µσ2
)
−
γ
n2
For the derivation of E (γ̂) note that
2
MAST90083 Computational Statistics & Data Mining Bootstrap Methods
E
[
(xj − µ)2(x̄− µ)
]
=
1
n
E (xj − µ)
3
=
γ
n
E
[
(xj − µ)(x̄− µ)2
]
=
1
n
n∑
i=1
E
[
(xi − µ)(x̄− µ)2
]
= E(x̄− µ)3
γ
n2
Using these two relations we have
E (γ̂) = E (xi − x̄)
3
= E
{
(xi − µ)3 − 3(xi − µ)2(x̄− µ) + 3(xi − µ)(x̄− µ)2 − (x̄− µ)3
}
= γ
(
1−
3
n
+
2
n2
)
Using these expressions we get
b2 = E
(
θ̂1 − θ0
)
=
3
n2
(
µσ2 − γ
)
+
6γ
n3
−
2γ
n4
6. The bias of the estimator θ̂1 is of order 1/n
2 compared with a bias of order 1/n for θ̂
Question 2
1. The empirical estimator is given by
θ̂ = θ (F1) =
[∫
xdF1(x)
]3
=
[
1
n
n∑
i=1
xi
]3
= x̄3
2. The evaluation of the bias requires the computation of
E
(
θ̂
)
= µ3 +
3µσ2
n
and b1 = E
(
θ̂
)
− θ0 =
3µσ2
n
since the population is normal γ = 0.
3. The bootstrap bias-reduced estimate is
θ̂1 = θ (F1)− b̂1 = 2θ (F1)− E [θ(F2)|F1]
= x̄3 −
3x̄σ̂2
n
4. The bias b2 requires the derivation of the expression of E
(
θ̂1
)
b2 = µ
3 +
µσ2
n
−
3
n
[
µσ2 −
1
n
µσ2
]
− µ3 =
3µσ2
n2
3
MAST90083 Computational Statistics & Data Mining Bootstrap Methods
5. In the case we use σ̃2 instead of σ̂2 we have
σ̃2 =
1
n− 1
n∑
i=1
(xi − x̄)
2
=
n
n− 1
σ̂2
and
θ̂1 = x̄
3 −
3x̄σ̃2
n
= x̄3 −
3x̄σ̂2
n− 1
and the bias is given by
b2 = µ
3 +
µσ2
n
−
3
n− 1
[
µσ2 −
1
n
µσ2
]
− µ3 = 0
Question 3
Let qα be the α−percentile of the bootstrap distribution
P
(
θ̂∗ − θ̂ ≤ qα
)
= P
(
θ̂∗ ≤ θ̂ + qα
)
= α
and denote θ̂∗α = θ̂ + qα. On the other hand let q1−α be the (1 − α)−percentile of the
bootstrap distribution
P
(
θ̂∗ − θ̂ ≤ q(1−α)
)
= P
(
θ̂∗ ≤ θ̂ + q(1−α)
)
= 1− α
and denote θ̂∗(1−α) = θ̂ + q(1−α). The relation
Ĥ−1(α) ≤ θ̂ − θ ≤ Ĥ−1(1− α)
corresponds to
1− 2α = P
(
qα ≤ θ̂ − θ ≤ q(1−α)
)
= P
(
θ̂∗α − θ̂ ≤ θ̂ − θ ≤ θ̂
∗
(1−α) − θ̂
)
= P
(
2θ̂ − θ̂∗α ≥ θ ≥ 2θ̂ − θ̂
∗
(1−α)
)
4