CS代考 Nonlinear Econometrics for Finance Lecture 5

Nonlinear Econometrics for Finance Lecture 5
. Econometrics for Finance Lecture 5 1 / 19

Maximum Likelihood Estimation (MLE)

Copyright By PowCoder代写 加微信 powcoder

. Econometrics for Finance Lecture 5 2 / 19

An example: The IID Normal case
Assume x1, x2, …, xT is an IID sample from a normal random variable with a certain μ and a certain σ2. Then, the likelihood is the joint probability of the sample:
2 􏰗T 1 −(xt−μ)2
L({x}, μ, σ ) = p(x1, x2, …, xT ) = p(x1)p(x2)…p(xT ) = The log-likelihood becomes:
2 􏰈T􏰃1−(xt−μ)2􏰄 log(L({x},μ,σ )) = log √ e σ2
√2πσ2 e 2σ2 .
T T2􏰈T(xt−μ)2
= −2 log(2π)− 2 log(σ )−
2σ2 . 1 2 1 121􏰈T(xt−μ)2
Dividing now by T:
T log(L({x},μ,σ )) = −2log(2π)−2log(σ )−T 2σ2 .
t=1 . Econometrics for Finance Lecture 5

The likelihood principle
Simple logic: Choose the parameters (μ and σ2, in this case) which make the sample more likely to have occurred.
Criterion: We maximize the (standardized by T, logarithmic) joint probability of the sample.
or, equivalently, in our example:
2􏰎12􏰏 = arg max log(L({x}, μ, σ )) ,
2 1 1 2 μ􏰉,σ􏰉 =argmax − log(2π)− log(σ )−
1 􏰈 (xt − μ)2
. Econometrics for Finance Lecture 5

log L(x, θ) = T
log p(xt|xt−1, …, θ) = log p(xt|xt−1, …, θ). T t=1 Tt=1
From IID to dependent data
Consider a sample of stationary observations xt , for t = 1, …, T . The joint probability of the sample is, again,
L(x,θ) = p(xT ,xT−1,xT−2,…,x1,θ).
Now, notice that
p(xT , xT −1, xT −2, …, x1, θ) = p(xT |xT −1, xT −2, …, x1, θ)p(xT −1, xT −2, …, x1, θ).
Similarly,
p(xT −1, xT −2, …, x1, θ) = p(xT −1|xT −2, …, x1, θ)p(xT −2, …, x1, θ).
Similarly, … we could keep on doing this.
Thus, the likelihood function for our stationary sample is the product of the conditional probabilities:
L(x, θ) = p(xT |xT −1, …, θ)p(xT −1|xT −2, …, θ)…p(x1, θ) = 􏰗 p(xt|xt−1, …, θ).
t=1 The (standardized by T, log) likelihood function becomes
. Econometrics for Finance Lecture 5

The maximum likelihood (ML) estimator
The maximum likelihood estimator θT is the maximizer of the ( 1 ) T
log-likelihood:
θT = arg max [QT (θ)] = arg max θθ
log p(xt|xt−1, …, θ)
Therefore, θT solves the following system of equations:
􏰉 = 􏰈 ∂QT (θT ) 1 T
􏰕 1 􏰈T T t=1
∂ log p(xt|xt−1, …, θT )
Nonlinear Econometrics for Finance Lecture 5

The score is the derivative of the log-conditional probability density with respect to θ evaluated at the true parameter value θ0 :
Theorem 1: We have
Et−1[scoret] = Et−1 ∂θ
conditional probabilities sum up to 1
􏰘 ∂p(xt|xt−1, …, θ0)
∂θ dxt = 0.
∂p(xt|xt−1, …, θ0) ∂p(xt|xt−1, …, θ0)
scoret = ∂logp(xt|xt−1,…,θ0) ∂θ
􏰎∂logp(xt|xt−1,…,θ0)􏰏
Proof: Note that Et−1[scoret]
= ∂θ p(xt|xt−1,…,θ0)p(xt|xt−1,…,θ0)dxt
∂ log p(xt|xt−1, …, θ0) ∂θ
p(xt|xt−1, …, θ0)dxt 1
p(xt|xt−1, …, θ0)dxt = 1
. Econometrics for Finance Lecture 5

We have shown that
􏰎∂ log p(xt|xt−1, …, θ0)􏰏 Et−1 ∂θ
Implications:
The score’s conditional and unconditional (by the law of iterated expectations) expectations are equal to zero.
The score is unforecastable.
Maximum likelihood is GMM on the score!
. Econometrics for Finance Lecture 5 8 / 19

Asymptotics
What happens when T → ∞?
The criterion:
log p(xt|xt−1, …, θ).
QT (θ) = T
Taking a Taylor’s expansions, stopped at the first order (around θ0):
∂ Q ( θ􏰉 ) ∂ Q ( θ ) ∂ 2 Q ( θ ) 􏰁 􏰂
T T − T 0 ≈ T 0 θT−θ0 .
d×1 vector Note, again, that
∂ log p(xt|xt−1, …, θT )
because θT is the solution of the maximization of QT (θ).
. Econometrics for Finance Lecture 5 9 / 19

Asymptotics
What happens when T → ∞?
Therefore, after re-arranging as in GMM, we are left with
􏰃∂2QT (θ0)􏰄−1 ∂QT (θ0) θT − θ0 = − .
∂θ∂θ⊤ ∂θ In the next few slides, we will prove that:
The MLE estimator is a consistent estimator: 􏰉
θ T →p θ 0 .
The MLE estimator is asymptotically normal:
T(θT −θ0)→N 0,B Ω0B . 00
. Econometrics for Finance Lecture 5

Consistency of the MLE estimator
 ∂2QT (θ0)  ∂QT (θ0) p
θT −θ0 =−
􏰌 􏰋􏰊 􏰍 􏰌 􏰋􏰊 􏰍
(b)→p B0 (a)→p 0
 ∂θ∂θ ∂θ
Using the WLLN: ∂2QT(θ0)
(b) = ∂θ∂θ⊤ Using the WLLN:
Because of Theorem 1
1 􏰈T ∂2logp(xt|xt−1,…,θ0) p 􏰃∂2logp(xt|xt−1,…,θ0)􏰄
= T t=1 ∂θ∂θ⊤
→ E ∂θ∂θ⊤ = B0.
∂QT (θ0) (a) = =
p 􏰃 ∂ log p(xt|xt−1, …, θ0) 􏰄
1 􏰈T ∂ log p(xt|xt−1, …, θ0) ∂θTt=1∂θ ∂θ
. Econometrics for Finance Lecture 5 11 / 19

Asymptotic normality of the MLE estimator
Using the WLLN: ∂2QT(θ0)
(b) = ∂θ∂θ⊤ Using the CLT:
1 􏰈T ∂2logp(xt|xt−1,…,θ0) p
􏰃∂2logp(xt|xt−1,…,θ0)􏰄 ∂θ∂θ⊤
T(θT −θ0)=− 
= T t=1 √T ∂QT (θ0)
1 􏰈T ∂ log p(xt|xt−1, …, θ0)
∂ QT(θ0) ∂QT(θ0) d −1 −1
T →N(0,B Ω0B ). ⊤00
􏰌 􏰋􏰊 􏰍 􏰌
(b)→p B0 (c)→d N(0,Ω0)
􏰁 ∂ log p(xt |xt−1 ,…,θ0 ) ∂ log p(xt |xt−1 ,…,θ0 ) 􏰂
where Ω0 = E ∂θ ∂θ⊤ .
. Econometrics for Finance Lecture 5

Asymptotic Normality of the MLE estimator
Let us understand term (c) better: √ ∂QT (θ0)
√ 1 􏰈 ∂ log p(xt|xt−1, …, θ0) = T Tt=1 ∂θ
 √ 1 􏰈T ∂logp(xt|xt−1,…,θ0) 􏰃∂logp(xt|xt−1,…,θ0)􏰄
=TTt=1 ∂θ −E ∂θ  􏰌 􏰋􏰊 􏰍
→d N (0, Ω0 ),
􏰁 ∂ log p(xt |xt−1 ,…,θ0 ) ∂ log p(xt |xt−1 ,…,θ0 ) 􏰂
with Ω0 = E ∂θ ∂θ⊤ .
=0 (by Theorem 1)
. Econometrics for Finance Lecture 5 13 / 19

The asymptotic variance: two equivalent expressions
Theorem 2: We have
􏰎 ∂ log p(xt|xt−1, …, θ0) ∂ log p(xt|xt−1, …, θ0) 􏰏 􏰕 ∂2 log p(xt|xt−1, …, θ0) 􏰖
⊤ =−E ⊤ . ∂θ∂θ
Proof: Remember that
Et−1 [scoret] =
∂ log p(xt|xt−1, …, θ0) ∂θ
p(xt|xt−1,…,θ0)dxt = 0.
Taking derivatives of both sides with respect to θ:
∂ 􏰘 ∂ log p(xt|xt−1, …, θ0)
∂θ⊤ ∂θ p(xt|xt−1, …, θ0)dxt
∂2 log p(xt|xt−1, …, θ0)
⊤ p(xt |xt−1 , …, θ0 )dxt ∂ log p(xt|xt−1, …, θ0) ∂p(xt|xt−1, …, θ0)
∂θ ∂θ⊤ dxt
∂2 log p(xt|xt−1, …, θ0)
⊤ p(xt |xt−1 , …, θ0 )dxt
∂ log p(xt|xt−1, …, θ0) ∂ log p(xt|xt−1, …, θ0)
p(xt|xt−1, …, θ0) 􏰌 􏰋􏰊 􏰍
=(a) (by the properties of the logarithm, as in Theorem 1)
B0+Ω0. . Bandi
Nonlinear Econometrics for Finance Lecture 5

The asymptotic variance: two equivalent expressions
Therefore,
(since Ω0 = −B0)
= N(0, B−1(−B0)B−1) 00
= N(0,−B−1) 0
􏰅 􏰃 ∂2 log p(xt|xt−1, …, θ0) 􏰄−1􏰆 = N 0,E − ∂θ∂θ⊤
= N(0,Ω−1) 0
􏰃 ∂ log p(xt|xt−1, …, θ0) ∂ log p(xt|xt−1, …, θ0) 􏰄−1􏰆 =N0,E ∂θ ∂θ⊤ .
→d N(0,B−1Ω0B−1)
. Econometrics for Finance Lecture 5 15 / 19

Estimating the MLE asymptotic variance: two equivalent expressions
1 􏰅 1 T ∂ log p(xt|xt−1, …, θT ) ∂ log p(xt|xt−1, …, θT )􏰆−1
T Tt=1 ∂θ ∂θ⊤
T Tt=1 ∂θ∂θ⊤
1 1􏰈∂2logp(x|x ,…,θ􏰉) V(θT)=− t t−1 T
. Econometrics for Finance Lecture 5 16 / 19

Estimation
= max 1 log(L({x}, μ, σ2)) θ=(μ,σ2) T
1 121􏰈T(xt−μ)2
∂QT (θT ) ∂ θ
This implies that:
Returning to our example: the IID normal case
= max2 − log(2π)− log(σ )−
θ=(μ,σ)2 2 T 2σ
The solution:
1 􏰇T (xt−μ􏰉T) 􏰆
24 ∂ σ2 2σ􏰉T T t=1 2σ􏰉T
􏰅∂QT(μ􏰉T,σ􏰉T)􏰆 􏰅
2 = 1 1􏰇T (xt−μ􏰉)2 =0. ∂ Q T ( μ􏰉 T , σ􏰉 T ) − + T
􏰃μ􏰉T􏰄 􏰃 1􏰇T xt 􏰄 􏰉 Tt=1
θT = 2 = 1 􏰇T 2 .
σ􏰉T T t=1(xt −μ􏰉T)
1 The ML estimators are (very intuitive) sample analogues to μ and σ2.
2 They are consistent when T → ∞. 222
3 However, σ􏰉T is biased (because we are dividing by T ): E(σ􏰉T ) ̸= σ .
. Econometrics for Finance Lecture 5 17 / 19

Returning to our example: the IID normal case
1 􏰅1 T ∂logp(xt,θT)∂logp(xt,θT)􏰆−1 V(θT)= 􏰈 􏰉 􏰉 ,
T Tt=1 ∂θ ∂θ⊤
∂ log p(xt, θT ) ∂θ
 (xt−μ􏰉T)  􏰉2
−1 +(xt−μ􏰉T) 24
2σ􏰉T 2σ􏰉T 􏰅T 􏰆−1
􏰈2􏰉 V(θT)= 1 −1 ∂ logp(xt,θT) ,
σ􏰉T 2σ􏰉T σ􏰉T
Notice the use of unconditional (rather than conditional) probabilities above. Why?
Because the sample is IID.
T T t=1 ∂θ∂θ⊤
−1 −(xt−μ􏰉T) 
􏰉24 ∂ logp(xt,θT) σ􏰉 σ􏰉
=T T2. ∂θ∂θ⊤ −(xt−μ􏰉T ) 1 − (xt−μ􏰉T )
. Econometrics for Finance Lecture 5 18 / 19

Let us see the MLE code for our Normal example …
. Econometrics for Finance Lecture 5 19 / 19

程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com