CS计算机代考程序代写 Chapter 6 Maximum Likelihood Methods

Chapter 6 Maximum Likelihood Methods
6.3 Maximum Likelihood Tests
1/20
Boxiang Wang
Chapter 6 STAT 4101 Spring 2021

Likelihood Ratio Test
2/20

Likelihood Ratio Test (LRT)
􏰉 Assume that X1,…,Xn is a random sample from the distribution for which the pdf is f (x; θ), θ ∈ Ω. Assume for now that θ is a scalar.
􏰉 Recall the likelihood function is
n
L(θ;x) = 􏰅f(xi;θ), θ ∈ Ω,
i=1
where x = (x1,…,xn); L is a function of θ.
􏰉 The log likelihood is
n
l(θ;x) = 􏰄logf(xi;θ), θ ∈ Ω.
i=1
3/20
Boxiang Wang
Chapter 6 STAT 4101 Spring 2021

Likelihood Ratio Test (LRT) cont’d
􏰉 Consider the two sided hypotheses
H0 :θ=θ0 versus H1 :θ̸=θ0,
where θ0 is a specified value.
􏰉 Let θˆ denote the mle of θ.
􏰉 Let
Λ = L(θ0). L(θˆ)
􏰉 Likelihood ratio test (LRT):
Reject H0 if Λ ≤ c,
where c is such that α = Pθ0 [Λ ≤ c].
􏰉 Motivation: recall Theorem 6.1.1: if θ0 is the true value of θ, then asymptotically L(θ0) is the maximum value of L(θ).
4/20
Boxiang Wang
Chapter 6 STAT 4101 Spring 2021

Example (6.3.2): LRT for the Mean of a Normal pdf
Consider a random sample X1, X2, . . . , Xn from a N(θ, σ2) distribution where −∞ < θ < ∞ and σ2 > 0 is known. Consider the hypothesis
H0 :θ=θ0 vsH1 :θ̸=θ0, where θ0 is specified. Show the likelihood ratio test.
5/20
Boxiang Wang
Chapter 6 STAT 4101 Spring 2021

The likelihood function L(θ) is 􏰍􏰎n/2􏰈 n 􏰡
1 exp −􏰀2σ2􏰁−1􏰄(xi−θ)2
=
i=1
1 exp −􏰀2σ2􏰁−1 􏰄(xi −x)2 exp −􏰀2σ2􏰁−1 n(x−θ)2 .
2πσ2
􏰍􏰎n/2􏰈n􏰡􏰤 􏰥
2πσ2
The mle is θˆ = X ̄ , and thus
i=1
Λ= L(θ0) =exp􏰤−􏰀2σ2􏰁−1n􏰀X−θ0􏰁2􏰥. 􏰛
L(θ)
LRT statistic: χ2L = −2 log Λ = σ/√n ∼ χ2(1).
􏰍 X − θ0 􏰎2
Thus the likelihood ratio test with significant level α states that we
reject H0 in favor of H1 when
χ2L =−2logΛ= σ/√n ≥χ2α(1).
􏰍 X − θ0 􏰎2
6/20
Boxiang Wang
Chapter 6 STAT 4101 Spring 2021

Theorem 6.3.1
Assume regularity conditions. Under the null hypothesis H0 : θ = θ0,
−2logΛ→D χ2(1).
Remark: Denote χ2Λ = −2 log Λ. For the hypotheses, this theorem
suggests the decision rule
Reject H0 if χ2Λ ≥ χ2α(1).
7/20
Boxiang Wang
Chapter 6 STAT 4101 Spring 2021

Wald Test
8/20

https://en.wikipedia.org/wiki/Abraham_Wald
9/20
Boxiang Wang
Chapter 6 STAT 4101 Spring 2021

Recall the mle θˆ: √ˆD􏰍1􏰎ˆP
n(θ−θ0)→N 0,I(θ0) ,andI(θ)→I(θ0). By Slutsky’s theorem,
􏰔ˆˆD
nI(θ)(θ − θ0) → N (0, 1) .
Wald-type test statistic:
2 􏰂􏰔 􏰋 􏰌􏰏2D2
χW= nI(θ) θ−θ0 →χ(1). Decision rule: Reject H0 in favor of H1 if χ2W ≥ χ2α(1).
􏰛􏰛
10/20
Boxiang Wang
Chapter 6 STAT 4101 Spring 2021

Example: Wald Test for the Mean of a Normal pdf
Consider a random sample X1, X2, . . . , Xn from a N(θ, σ2) with known σ. Consider the hypothesis
H0 :θ=θ0 vsH1 :θ̸=θ0. Show the Wald test.
􏰉θˆ =X ̄. MLE
􏰉 I(θ0) = 1/σ2.
􏰉 Wald test statistic:
2 􏰂􏰔 􏰋 􏰌􏰏2 􏰛􏰛
χW= nI(θ) θ−θ0
􏰍 X − θ0 􏰎2
= σ/√n ∼ χ2(1).
11/20
Boxiang Wang
Chapter 6 STAT 4101 Spring 2021

Score Test
12/20

https://en.wikipedia.org/wiki/C._R._Rao
13/20
Boxiang Wang
Chapter 6 STAT 4101 Spring 2021

Score function:
􏰍∂ log f (X1; θ) ∂ log f (Xn; θ)􏰎⊤ S(θ) = ∂θ ,…, ∂θ .
We also have:
l′(θ0)=􏰄n ∂logf(Xi;θ0)andVar(l′(θ0))=nI(θ).
i=1 ∂θ Score-type test statistic:
􏰐′ 􏰑2
2 l(θ0)D2
χR = 􏰓nI(θ0) →χ (1). Decision rule: Reject H0 in favor of H1 if χ2R ≥ χ2α(1).
14/20
Boxiang Wang
Chapter 6 STAT 4101 Spring 2021

Example: Score Test for the Mean of a Normal pdf
Consider a random sample X1, X2, . . . , Xn from a N(θ, σ2) with known σ. Consider the hypothesis
H0 :θ=θ0 vsH1 :θ̸=θ0. Show the score test.
􏰉θˆ =X ̄. MLE
􏰉 I(θ0) = 1/σ2. 􏰉l′(θ)=􏰄n Xi−θ0.
i=1 σ2 􏰉 Score test statistic:
􏰐 l′ (θ0) 􏰑2 χ2R= 􏰓nI(θ0)
􏰍 X − θ0 􏰎2
= σ/√n ∼ χ2(1).
15/20
Boxiang Wang
Chapter 6 STAT 4101 Spring 2021

Three Types of Maximum Likelihood Tests
􏰉 In the example of the test for the mean of a normal pdf, all three types of test are equivalent, and the distribution of the test statistic are all exactly χ2(1).
􏰉 In most examples, the three tests are not equivalently but asymptotically equivalently under the null hypothesis.
􏰉 When n → ∞, all three test statistics have a limiting distribution of χ2(1):
LRT: Waldtest: Scoretest:
χ2L=−2logL(θ0)→D χ2(1). L(θˆ)
2 􏰂􏰔 􏰋 􏰌􏰏2D2 􏰛􏰛
χW = nI(θ) θ−θ0 →χ (1). 􏰐′ 􏰑2
2 l(θ0)D2
χR = 􏰓nI(θ0) →χ (1).
16/20
Boxiang Wang
Chapter 6 STAT 4101 Spring 2021

Example (6.2.4)
Let X1,X2,…,Xn denote a random sample of size n > 2 from a distribution with pdf
􏰂 θxθ−1 for0