CS代写 Nonlinear Econometrics for Finance Lecture 5

Nonlinear Econometrics for Finance Lecture 5
. Econometrics for Finance Lecture 5 1 / 19

Maximum Likelihood Estimation (MLE)

. Econometrics for Finance Lecture 5 2 / 19

An example: The IID Normal case
Assume x1, x2, …, xT is an IID sample from a normal random variable with a certain μ and a certain σ2. Then, the likelihood is the joint probability of the sample:
2 􏰎T 1 −(xt−μ)2
L({x}, μ, σ ) = p(x1, x2, …, xT ) = p(x1)p(x2)…p(xT ) = The log-likelihood becomes:
2 􏰍T􏰬1−(xt−μ)2􏰭 log(L({x},μ,σ )) = log √ e σ2
√2πσ2 e 2σ2 .
T T2􏰍T(xt−μ)2
= −2 log(2π)− 2 log(σ )−
2σ2 . 1 2 1 121􏰍T(xt−μ)2
Dividing now by T:
T log(L({x},μ,σ )) = −2log(2π)−2log(σ )−T 2σ2 .
t=1 . Econometrics for Finance Lecture 5

The likelihood principle
Simple logic: Choose the parameters (μ and σ2, in this case) which make the sample more likely to have occurred.
Criterion: We maximize the (standardized by T, logarithmic) joint probability of the sample.
or, equivalently, in our example:
2􏰮12􏰯 = arg max log(L({x}, μ, σ )) ,
2 1 1 2 μ􏰑,σ􏰑 =argmax − log(2π)− log(σ )−
1 􏰍 (xt − μ)2
. Econometrics for Finance Lecture 5

log L(x, θ) = T
log p(xt|xt−1, …, θ) = log p(xt|xt−1, …, θ). T t=1 Tt=1
From IID to dependent data
Consider a sample of stationary observations xt , for t = 1, …, T . The joint probability of the sample is, again,
L(x,θ) = p(xT ,xT−1,xT−2,…,x1,θ).
Now, notice that
p(xT , xT −1, xT −2, …, x1, θ) = p(xT |xT −1, xT −2, …, x1, θ)p(xT −1, xT −2, …, x1, θ).
Similarly,
p(xT −1, xT −2, …, x1, θ) = p(xT −1|xT −2, …, x1, θ)p(xT −2, …, x1, θ).
Similarly, … we could keep on doing this.
Thus, the likelihood function for our stationary sample is the product of the conditional probabilities:
L(x, θ) = p(xT |xT −1, …, θ)p(xT −1|xT −2, …, θ)…p(x1, θ) = 􏰎 p(xt|xt−1, …, θ).
t=1 The (standardized by T, log) likelihood function becomes
. Econometrics for Finance Lecture 5

The maximum likelihood (ML) estimator
The maximum likelihood estimator θT is the maximizer of the ( 1 ) T
log-likelihood:
θT = arg max [QT (θ)] = arg max θθ
log p(xt|xt−1, …, θ)
Therefore, θT solves the following system of equations:
􏰑 = 􏰍 ∂QT (θT ) 1 T
􏰃 1 􏰍T T t=1
∂ log p(xt|xt−1, …, θT )
Nonlinear Econometrics for Finance Lecture 5

The score is the derivative of the log-conditional probability density with respect to θ evaluated at the true parameter value θ0 :
Theorem 1: We have
Et−1[scoret] = Et−1 ∂θ
conditional probabilities sum up to 1
􏰏 ∂p(xt|xt−1, …, θ0)
∂θ dxt = 0.
∂p(xt|xt−1, …, θ0) ∂p(xt|xt−1, …, θ0)
scoret = ∂logp(xt|xt−1,…,θ0) ∂θ
􏰮∂logp(xt|xt−1,…,θ0)􏰯
Proof: Note that Et−1[scoret]
= ∂θ p(xt|xt−1,…,θ0)p(xt|xt−1,…,θ0)dxt
∂ log p(xt|xt−1, …, θ0) ∂θ
p(xt|xt−1, …, θ0)dxt 1
p(xt|xt−1, …, θ0)dxt = 1
. Econometrics for Finance Lecture 5

We have shown that
􏰮∂ log p(xt|xt−1, …, θ0)􏰯 Et−1 ∂θ
Implications:
The score’s conditional and unconditional (by the law of iterated expectations) expectations are equal to zero.
The score is unforecastable.
Maximum likelihood is GMM on the score!
. Econometrics for Finance Lecture 5 8 / 19

Asymptotics
What happens when T → ∞?
The criterion:
log p(xt|xt−1, …, θ).
QT (θ) = T
Taking a Taylor’s expansions, stopped at the first order (around θ0):
∂ Q ( θ􏰑 ) ∂ Q ( θ ) ∂ 2 Q ( θ ) 􏰪 􏰫
T T − T 0 ≈ T 0 θT−θ0 .
d×1 vector Note, again, that
∂ log p(xt|xt−1, …, θT )
because θT is the solution of the maximization of QT (θ).
. Econometrics for Finance Lecture 5 9 / 19

Asymptotics
What happens when T → ∞?
Therefore, after re-arranging as in GMM, we are left with
􏰬∂2QT (θ0)􏰭−1 ∂QT (θ0) θT − θ0 = − .
∂θ∂θ⊤ ∂θ In the next few slides, we will prove that:
The MLE estimator is a consistent estimator: 􏰑
θ T →p θ 0 .
The MLE estimator is asymptotically normal:
T(θT −θ0)→N 0,B Ω0B . 00
. Econometrics for Finance Lecture 5

Consistency of the MLE estimator
 ∂2QT (θ0)  ∂QT (θ0) p
θT −θ0 =−
􏱦 􏱥􏱤 􏱧 􏱦 􏱥􏱤 􏱧
(b)→p B0 (a)→p 0
 ∂θ∂θ ∂θ
Using the WLLN: ∂2QT(θ0)
(b) = ∂θ∂θ⊤ Using the WLLN:
Because of Theorem 1
1 􏰍T ∂2logp(xt|xt−1,…,θ0) p 􏰬∂2logp(xt|xt−1,…,θ0)􏰭
= T t=1 ∂θ∂θ⊤
→ E ∂θ∂θ⊤ = B0.
∂QT (θ0) (a) = =
p 􏰬 ∂ log p(xt|xt−1, …, θ0) 􏰭
1 􏰍T ∂ log p(xt|xt−1, …, θ0) ∂θTt=1∂θ ∂θ
. Econometrics for Finance Lecture 5 11 / 19

Asymptotic normality of the MLE estimator
Using the WLLN: ∂2QT(θ0)
(b) = ∂θ∂θ⊤ Using the CLT:
1 􏰍T ∂2logp(xt|xt−1,…,θ0) p
􏰬∂2logp(xt|xt−1,…,θ0)􏰭 ∂θ∂θ⊤
T(θT −θ0)=− 
= T t=1 √T ∂QT (θ0)
1 􏰍T ∂ log p(xt|xt−1, …, θ0)
∂ QT(θ0) ∂QT(θ0) d −1 −1
T →N(0,B Ω0B ). ⊤00
􏱦 􏱥􏱤 􏱧 􏱦
(b)→p B0 (c)→d N(0,Ω0)
􏰪 ∂ log p(xt |xt−1 ,…,θ0 ) ∂ log p(xt |xt−1 ,…,θ0 ) 􏰫
where Ω0 = E ∂θ ∂θ⊤ .
. Econometrics for Finance Lecture 5

Asymptotic Normality of the MLE estimator
Let us understand term (c) better: √ ∂QT (θ0)
√ 1 􏰍 ∂ log p(xt|xt−1, …, θ0) = T Tt=1 ∂θ
 √ 1 􏰍T ∂logp(xt|xt−1,…,θ0) 􏰬∂logp(xt|xt−1,…,θ0)􏰭
=TTt=1 ∂θ −E ∂θ  􏱦 􏱥􏱤 􏱧
→d N (0, Ω0 ),
􏰪 ∂ log p(xt |xt−1 ,…,θ0 ) ∂ log p(xt |xt−1 ,…,θ0 ) 􏰫
with Ω0 = E ∂θ ∂θ⊤ .
=0 (by Theorem 1)
. Econometrics for Finance Lecture 5 13 / 19

The asymptotic variance: two equivalent expressions
Therefore,
(since Ω0 = −B0)
= N(0, B−1(−B0)B−1) 00
= N(0,−B−1) 0
􏰳 􏰬 ∂2 log p(xt|xt−1, …, θ0) 􏰭−1􏰂 = N 0,E − ∂θ∂θ⊤
= N(0,Ω−1) 0
􏰬 ∂ log p(xt|xt−1, …, θ0) ∂ log p(xt|xt−1, …, θ0) 􏰭−1􏰂 =N0,E ∂θ ∂θ⊤ .
→d N(0,B−1Ω0B−1)
. Econometrics for Finance Lecture 5 15 / 19

Estimating the MLE asymptotic variance: two equivalent expressions
1 􏰳 1 T ∂ log p(xt|xt−1, …, θT ) ∂ log p(xt|xt−1, …, θT )􏰂−1
T Tt=1 ∂θ ∂θ⊤
T Tt=1 ∂θ∂θ⊤
1 1􏰍∂2logp(x|x ,…,θ􏰑) V(θT)=− t t−1 T
. Econometrics for Finance Lecture 5 16 / 19

Estimation
= max 1 log(L({x}, μ, σ2)) θ=(μ,σ2) T
1 121􏰍T(xt−μ)2
∂QT (θT ) ∂ θ
This implies that:
Returning to our example: the IID normal case
= max2 − log(2π)− log(σ )−
θ=(μ,σ)2 2 T 2σ
The solution:
1 􏰉T (xt−μ􏰑T) 􏰂
24 ∂ σ2 2σ􏰑T T t=1 2σ􏰑T
􏰳∂QT(μ􏰑T,σ􏰑T)􏰂 􏰳
2 = 1 1􏰉T (xt−μ􏰑)2 =0. ∂ Q T ( μ􏰑 T , σ􏰑 T ) − + T
􏰬μ􏰑T􏰭 􏰬 1􏰉T xt 􏰭 􏰑 Tt=1
θT = 2 = 1 􏰉T 2 .
σ􏰑T T t=1(xt −μ􏰑T)
1 The ML estimators are (very intuitive) sample analogues to μ and σ2.
2 They are consistent when T → ∞. 222
3 However, σ􏰑T is biased (because we are dividing by T ): E(σ􏰑T ) ̸= σ .
. Econometrics for Finance Lecture 5 17 / 19

Returning to our example: the IID normal case
1 􏰳1 T ∂logp(xt,θT)∂logp(xt,θT)􏰂−1 V(θT)= 􏰍 􏰑 􏰑 ,
T Tt=1 ∂θ ∂θ⊤
∂ log p(xt, θT ) ∂θ
 (xt−μ􏰑T)  􏰑2
−1 +(xt−μ􏰑T) 24
2σ􏰑T 2σ􏰑T 􏰳T 􏰂−1
􏰍2􏰑 V(θT)= 1 −1 ∂ logp(xt,θT) ,
σ􏰑T 2σ􏰑T σ􏰑T
Notice the use of unconditional (rather than conditional) probabilities above. Why?
Because the sample is IID.
T T t=1 ∂θ∂θ⊤
−1 −(xt−μ􏰑T) 
􏰑24 ∂ logp(xt,θT) σ􏰑 σ􏰑
=T T2. ∂θ∂θ⊤ −(xt−μ􏰑T ) 1 − (xt−μ􏰑T )
. Econometrics for Finance Lecture 5 18 / 19

Let us see the MLE code for our Normal example …
. Econometrics for Finance Lecture 5 19 / 19

程序代写 CS代考加微信: powcoder QQ: 1823890830 Email: powcoder@163.com

Related Posts