Chapter 6 Maximum Likelihood Methods
6.3 Maximum Likelihood Tests
1/20
Boxiang Wang
Chapter 6 STAT 4101 Spring 2021
Likelihood Ratio Test
2/20
Likelihood Ratio Test (LRT)
Assume that X1,…,Xn is a random sample from the distribution for which the pdf is f (x; θ), θ ∈ Ω. Assume for now that θ is a scalar.
Recall the likelihood function is
n
L(θ;x) = f(xi;θ), θ ∈ Ω,
i=1
where x = (x1,…,xn); L is a function of θ.
The log likelihood is
n
l(θ;x) = logf(xi;θ), θ ∈ Ω.
i=1
3/20
Boxiang Wang
Chapter 6 STAT 4101 Spring 2021
Likelihood Ratio Test (LRT) cont’d
Consider the two sided hypotheses
H0 :θ=θ0 versus H1 :θ̸=θ0,
where θ0 is a specified value.
Let θˆ denote the mle of θ.
Let
Λ = L(θ0). L(θˆ)
Likelihood ratio test (LRT):
Reject H0 if Λ ≤ c,
where c is such that α = Pθ0 [Λ ≤ c].
Motivation: recall Theorem 6.1.1: if θ0 is the true value of θ, then asymptotically L(θ0) is the maximum value of L(θ).
4/20
Boxiang Wang
Chapter 6 STAT 4101 Spring 2021
Example (6.3.2): LRT for the Mean of a Normal pdf
Consider a random sample X1, X2, . . . , Xn from a N(θ, σ2) distribution where −∞ < θ < ∞ and σ2 > 0 is known. Consider the hypothesis
H0 :θ=θ0 vsH1 :θ̸=θ0, where θ0 is specified. Show the likelihood ratio test.
5/20
Boxiang Wang
Chapter 6 STAT 4101 Spring 2021
The likelihood function L(θ) is n/2 n
1 exp −2σ2−1(xi−θ)2
=
i=1
1 exp −2σ2−1 (xi −x)2 exp −2σ2−1 n(x−θ)2 .
2πσ2
n/2n
2πσ2
The mle is θˆ = X ̄ , and thus
i=1
Λ= L(θ0) =exp−2σ2−1nX−θ02.
L(θ)
LRT statistic: χ2L = −2 log Λ = σ/√n ∼ χ2(1).
X − θ0 2
Thus the likelihood ratio test with significant level α states that we
reject H0 in favor of H1 when
χ2L =−2logΛ= σ/√n ≥χ2α(1).
X − θ0 2
6/20
Boxiang Wang
Chapter 6 STAT 4101 Spring 2021
Theorem 6.3.1
Assume regularity conditions. Under the null hypothesis H0 : θ = θ0,
−2logΛ→D χ2(1).
Remark: Denote χ2Λ = −2 log Λ. For the hypotheses, this theorem
suggests the decision rule
Reject H0 if χ2Λ ≥ χ2α(1).
7/20
Boxiang Wang
Chapter 6 STAT 4101 Spring 2021
Wald Test
8/20
https://en.wikipedia.org/wiki/Abraham_Wald
9/20
Boxiang Wang
Chapter 6 STAT 4101 Spring 2021
Recall the mle θˆ: √ˆD1ˆP
n(θ−θ0)→N 0,I(θ0) ,andI(θ)→I(θ0). By Slutsky’s theorem,
ˆˆD
nI(θ)(θ − θ0) → N (0, 1) .
Wald-type test statistic:
2 2D2
χW= nI(θ) θ−θ0 →χ(1). Decision rule: Reject H0 in favor of H1 if χ2W ≥ χ2α(1).
10/20
Boxiang Wang
Chapter 6 STAT 4101 Spring 2021
Example: Wald Test for the Mean of a Normal pdf
Consider a random sample X1, X2, . . . , Xn from a N(θ, σ2) with known σ. Consider the hypothesis
H0 :θ=θ0 vsH1 :θ̸=θ0. Show the Wald test.
θˆ =X ̄. MLE
I(θ0) = 1/σ2.
Wald test statistic:
2 2
χW= nI(θ) θ−θ0
X − θ0 2
= σ/√n ∼ χ2(1).
11/20
Boxiang Wang
Chapter 6 STAT 4101 Spring 2021
Score Test
12/20
https://en.wikipedia.org/wiki/C._R._Rao
13/20
Boxiang Wang
Chapter 6 STAT 4101 Spring 2021
Score function:
∂ log f (X1; θ) ∂ log f (Xn; θ)⊤ S(θ) = ∂θ ,…, ∂θ .
We also have:
l′(θ0)=n ∂logf(Xi;θ0)andVar(l′(θ0))=nI(θ).
i=1 ∂θ Score-type test statistic:
′ 2
2 l(θ0)D2
χR = nI(θ0) →χ (1). Decision rule: Reject H0 in favor of H1 if χ2R ≥ χ2α(1).
14/20
Boxiang Wang
Chapter 6 STAT 4101 Spring 2021
Example: Score Test for the Mean of a Normal pdf
Consider a random sample X1, X2, . . . , Xn from a N(θ, σ2) with known σ. Consider the hypothesis
H0 :θ=θ0 vsH1 :θ̸=θ0. Show the score test.
θˆ =X ̄. MLE
I(θ0) = 1/σ2. l′(θ)=n Xi−θ0.
i=1 σ2 Score test statistic:
l′ (θ0) 2 χ2R= nI(θ0)
X − θ0 2
= σ/√n ∼ χ2(1).
15/20
Boxiang Wang
Chapter 6 STAT 4101 Spring 2021
Three Types of Maximum Likelihood Tests
In the example of the test for the mean of a normal pdf, all three types of test are equivalent, and the distribution of the test statistic are all exactly χ2(1).
In most examples, the three tests are not equivalently but asymptotically equivalently under the null hypothesis.
When n → ∞, all three test statistics have a limiting distribution of χ2(1):
LRT: Waldtest: Scoretest:
χ2L=−2logL(θ0)→D χ2(1). L(θˆ)
2 2D2
χW = nI(θ) θ−θ0 →χ (1). ′ 2
2 l(θ0)D2
χR = nI(θ0) →χ (1).
16/20
Boxiang Wang
Chapter 6 STAT 4101 Spring 2021
Example (6.2.4)
Let X1,X2,…,Xn denote a random sample of size n > 2 from a distribution with pdf
θxθ−1 for0