The University of of Mathematics and Statistics
Solutions to Tutorial Week 13
STAT3023: Statistical Inference
Lecturers: and
Copyright By PowCoder代写 加微信 powcoder
1. Please complete any work still remaining from the Week 12 Tutorial. Review exercises based on the geometric distribution
Semester 2, 2021
2. Recall that if a discrete random variable X has probability mass function (PMF) in the exponential family form
Pθ(X = x) = eθt(x)−K(θ)−M(x) (1) then Eθ [t(X)] = K′(θ) and Varθ [t(X)] = K′′(θ). We call the parameter θ the “natural” or
“canonical” parameter of the exponential family.
Suppose X has a geometric(p) distribution so that P(X = x) = (1 − p)x−1p for x = 1,2,… By
writing the PMF of X in exponential family form (1), deduce E(X) and Var(X) as functions of p. Solution: Since
P(X=x)=(1−p)x−1p=exlog(1−p) p 1−p
we need to change parameters to θ = log(1−p); equivalently p = 1−eθ and 1−p = eθ. Then the PMF takes the form
xθ1−eθ xθ −θ θx−[−log(e−θ−1)] pθ(x)=e eθ =e e −1=e
which is in form (1) with
Differentiating once gives
K(θ) = − log e−θ − 1 .
′ −e−θ e−θ 1
K(θ)=− e−θ−1 =e−θ−1=1−eθ after multiplying top and bottom by eθ. Differentiating again gives
′′ θ−2 θ eθ
K (θ) = −(1 − e ) −e = (1 − eθ)2 .
Expressing these in terms of p we get
E(X)=1 andVar(X)=1−p.
3. Suppose X1,…,Xn are iid geometric with P(X1 = x) = (1 − p)x−1p for x = 1,2,… but it is
desired to estimate the natural/canonical parameter θ rather than p.
(a) Write down the likelihood fθ(X) in terms of the natural parameter θ and hence obtain the
score function l′(θ;X) = ∂ logfθ(X). ∂θ
Copyright © 2021 The University of Sydney
fθ(X)=eθXi(e−θ −1)=eθT e−θ −1n
where T = ni=1 Xi. Thus the log-likelihood and its derivative with respect to θ are (re- spectively)
l(θ;X)=logfθ(X)=Tθ+nloge−θ −1 ′ ne−θ n
l (θ; X) = T − e−θ − 1 = T − 1 − eθ .
(b) Determine the Cram ́er-Rao lower bound to the variance of an unbiased estimator of θ.
′ neθ Varθ[l(θ;X)]=Varθ(T)=nVarθ(X1)=(1−eθ)2 .
Therefore for any unbiased estimator θˆ of θ,
1 (1−eθ)2
Varθ θˆ ≥ Varθ [l′(θ;X)] = neθ (c) Derive the maximum-likelihood estimator θˆML of θ.
Setting the score function equal to zero and solving gives T=n
(d) Show, using the delta method, that θˆML is AMVU.
We have already shown above that E(X ) = 1
EθX ̄= 1 , 1−eθ
Varθ X =n(1−eθ)2.
and Var(X ) = 1
eθ . (1−eθ )2
1−eθ e θ = 1 − 1 ̄
ˆ1 θML=log 1−X ̄ .
In fact we have more: by the Central Limit Theorem
X ̄ ∼AN 1−eθ,n(1−eθ)2 .
Now the function
g(x) = log
has derivative
1 x−1 1− x = log x
= log(x−1)−log(x)
g′(x)= 1 −1= 1 . x−1 x x(x−1)
Thus by the delta method,
gX ̄∼AN g ,g′
112 eθ 1−eθ 1−eθ n(1−eθ)2
The asymptotic mean is
(unsurprisingly). Next,
so the asymptotic variance is ′ 1 2
′ 1 1−eθ2 g 1−eθ = eθ
eθ 1−eθ4 eθ 1−eθ2 g 1−eθ n(1−eθ)2 = e2θ n(1−eθ)2 = neθ .
Since this coincides with the Cram ́er-Rao lower bound, θˆML is AMVU.
4. Suppose X1,…,X10 are iid geometric with P(X = x) = (1 − p)x−1p. Derive the UMP test at level 0.05 for testing H0 : p = 0.5 against the alternative H1 : p > 0.5. You may use the R output below and the fact that T = ni=1 Xi has a negative binomial distribution and the CDF of T − n, specifically
P (T − n ≤ x) is given by the R function pnbinom(x,n,p).
> cbind(x,pnbinom(x,10,.5))
[1,] 0 0.0009765625
[2,] 1 0.0058593750
[3,] 2 0.0192871094
[4,] 3 0.0461425781
[5,] 4 0.0897827148
[6,] 5 0.1508789063
[7,] 6 0.2272491455
[8,] 7 0.3145294189
[9,] 8 0.4072647095
[10,] 9 0.5000000000
[11,] 10 0.5880985260
[12,] 11 0.6681880951
[13,] 12 0.7382664680
[14,] 13 0.7975635529
[15,] 14 0.8462718725
[16,] 15 0.8852385283
[17,] 16 0.9156812280
[18,] 17 0.9389609396
[19,] 18 0.9564207233
[20,] 19 0.9692858271
[21,] 20 0.9786130274
Solution: Fix p = p1 > 0.5. Then the (Neyman-Pearson) likelihood ratio statistic for testing H0:p=0.5againstH1:p=p1 is
fp1(X) n (1−p1)Xi−1p1 n 1−p1Xi p1 1−p1T p1 n
f (X) = 0.5
= 0.5 1 − p 1
The most powerful test, according to the Neyman-Pearson lemma rejects for large values of this statistic. Since 1−p1 < 1, this is an decreasing function of T and the most powerful test rejects for
small values of T. Specifically the test function is of the form
1 forT
where C and γ are chosen so that E0.5 [δ(X)] = 0.05. Since this doesn’t depend on the value of p1 it is UMP.
To determine the values C and γ, note that we must have
0.05=E[δ(X)]=P(T