The University of Queensland
(School of Economics) Applied Econometrics for Macro and Finance Week 3 1 / 33
Forecasting Univariate Processes – II
Copyright By PowCoder代写 加微信 powcoder
Properties of Polynomials in the Lag Operator
Writing the general ARMA(p, q) using the lag operator a(L)yt = a0 + b(L)εt
is very useful because:
we can treat a(L) and b(L) as polynomial functions;
polynomial rules and techniques can be applied (with very few exceptions) to analyse a(L) and b(L).
We can multiply and divide a(L) and b(L) using polynomial expansion and division. We can examine the roots of a(L) and b(L), which provide information on stability and
invertibility of the ARMA process.
Consequently, we can factor a(L) and b(L); this will be used later in the course.
(School of Economics) Applied Econometrics for Macro and Finance Week 3 2 / 33
Polynomial Expansion
Consider two general polynomials in L:
α(L)=α0 +α1L+…+αrLr,
β(L)=β0 +β1L+…+βsLs.
α(L) and β(L) can be multiplied together: θ(L) = α(L)β(L); θ(L) is a new polynomial of
degree r + s:
In general,
θ(L)=θ0 +θ1L+···+θr+sLr+s.
θj =α0βj +···+αjβ0, j=0,1,…,r+s,
whereαj =0forallj>randβj =0forallj>s.
(School of Economics) Applied Econometrics for Macro and Finance Week 3 3 / 33
Polynomial Expansion: AR Process with AR Errors
Consider an AR(1) model, yt = a0 + a1yt−1 + ut, where ut are themselves auto-regressive: ut = b1ut−1 + εt,
with εt mean-independent.
Using lag operator notation, this is a(L)yt = a0 + ut, b(L)ut = εt, where:
a(L) = 1 − a1L, b(L) = 1 − b1L. Show that this is equivalent to an AR(2):
b(L)a(L)yt = b(L)a0 + b(L)ut, a ( L ) y t = b ( L ) a 0 + ε t , a(L)yt = (1 − b1)a0 + εt.
(School of Economics) Applied Econometrics for Macro and Finance Week 3 4 / 33
Polynomial Expansion: AR Process with AR Errors
The product b(L)a(L) is applied to a time series variable yt, so we can use standard polynomial expansion to derive a(L):
a(L) = (1 − b1L)(1 − a1L) = a0 + a1L + a2L2,
a0 = (1)(1) = 1,
a1 = (1)(−a1) + (−b1)(1) = −(a1 + b1), a2 = (1)(0) + (−b1)(−a1) + (0)(1) = a1b1.
The product b(L)a0 is not applied to a time series variable (but instead a constant); this is equivalent to replacing L with 1:
b(L)a0 = b(1)a0 = (1 − (b1)(1))a0 = (1 − b1)a0.
The resulting AR(2) process is yt = (1 − b1)a0 + (a1 + b1)yt−1 − a1b1yt−2 + εt.
(School of Economics) Applied Econometrics for Macro and Finance Week 3 5 / 33
Polynomial Division
Polynomials α(L) and β(L) can sometimes be divided: θ(L) = α(L) . β(L)
In general, θ(L) will have the form:
θ(L) = θ0 + θ1L + θ2L2 + · · · ,
but it is not a polynomial because there are generally infinitely many terms.
The roots of β(L) (the denominator) determine if θj is finite or diverges as j −→ ∞.
Polynomial division is implemented using the method of undetermined coefficients. Rewrite the division as a multiplication: β(L)θ(L) = α(L).
Use expansion techniques on β(L)θ(L) to obtain an equation for each αj. Solve recursively for θj.
(School of Economics) Applied Econometrics for Macro and Finance Week 3 6 / 33
Method of Undetermined Coefficients
The general solution is obtained by solving:
(β0 +β1L+β2L2 +···+βsLs)(θ0 +θ1L+θ2L2 +···)=α0 +α1L+α2L2 +···+αrLr. The coefficients for each term Lj , j = 0, 1, 2, . . . , must match on both sides of the equality.
L0 : L1 : L2 :
β0θ0 = α0 β0θ1 +β1θ0 =α1 β0θ2 +β1θ1 +β2θ0 =α2
⇒ θ0 = α0/β0,
⇒ θ1 =(α1 −β1θ0)/β0,
⇒ θ2 =(α2 −β1θ1 −β2θ0)/β0,
The recursive solution is:
θ0=α0/β0, (School of Economics)
θj=αj− βlθj−l/β0forj≥1.
Applied Econometrics for Macro and Finance Week 3 7 / 33
Characteristic Roots
For any polynomial in the lag operator, α(L), we can formulate the characteristic equation by replacing L with a complex number, i.e. z ∈ C:
α(z)=α0+α1z+···+αrzr =0. (1) The roots of α(z)—that is, all values of z that solve (1)—are called the characteristic roots
(usually, referred to simply as roots) of α(L).
The roots of α(L) play an important role in division by α(L): θ(L) = 1 is one-sided,
α(L) Thisimpliesθ(L)=θ0+θ1L+θ2L2+··· withθj −→0asj−→∞.
convergent if and only if α(z) ̸= 0 for all |z| ≤ 1.
Stability and invertibility in a general ARMA(p, q) are defined by characteristic roots.
(School of Economics) Applied Econometrics for Macro and Finance Week 3 8 / 33
AR and MA Representations of the ARMA(p, q)
Consider the ARMA(p, q): a(L)yt = a0 + b(L)εt.
The ARMA(p, q) is stable if and only a(z) ̸= 0 for all |z| ≤ 1. Then, a(L) can be inverted to
produce a pure MA representation.
The quotient θ(L) = b(L)/a(L) is applied to a time series variable εt, so we can use standard
polynomial division to derive θ(L).
The quotient a0/a(L) is not applied to a time series variable (but instead a constant); hence,
replace L with 1:
a0/a(L)=a0/a(1)=a0/(1−a1 −···−ap). Consequently, the pure MA representation is: yt = a0/a(1) + θ(L)εt.
(School of Economics) Applied Econometrics for Macro and Finance Week 3 9 / 33
AR and MA Representations of the ARMA(p, q)
The ARMA(p, q) is invertible if and only b(z) ̸= 0 for all |z| ≤ 1. Then, b(L) can be inverted
to produce a pure AR representation.
The quotient φ(L) = a(L)/b(L) is applied to a time series variable yt, so we can use standard
polynomial division to derive φ(L).
The quotient a0/b(L) is not applied to a time series variable (but instead a constant); hence,
replace L with 1:
a0/b(L)=a0/b(1)=a0/(1−b1 −···−bq). Consequently, the pure AR representation is: φ(L)yt = a0/b(1) + εt.
If the ARMA(p, q) is stable and invertible, then we also refer to the MA representation yt = a0/a(1) + θ(L)εt as the Wold representation.
Every stationary stochastic process admits a Wold representation!
(School of Economics) Applied Econometrics for Macro and Finance Week 3 10 / 33
Example: MA Representation of an ARMA(2, 1)
Consider the ARMA(2, 1) model:
yt = 1.2yt−1 − 0.7yt−2 + εt + 0.3εt−1.
Inthiscase,a(L)=1−1.2L+0.7L2 andb(L)=1+0.3L. We wish to find
θ(L) = (1 + 0.3L)/(1 − 1.2L + 0.7L2) = 1 + θ1L + θ2L2 + · · · .
(School of Economics) Applied Econometrics for Macro and Finance Week 3 11 / 33
Example: MA Representation of an ARMA(2, 1)
1+θ1L+θ2L2 +···=(1+0.3L)/(1−1.2L+0.7L2), (1−1.2L+0.7L2)(1+θ1L+θ2L2 +···)=1+0.3L,
1 + θ1L + θ2L2 − 1.2L − 1.2θ1L2 − 1.2θ2L3−
+0.7L2 +0.7θ1L3 +0.7θ2L4 −···=1+0.3L,
which leads to
L0: L1: L2: L3: Lk:
1 = 1, θ1 − 1.2 = 0.3
θ2 −1.2θ1 +0.7=0 θ3 − 1.2θ2 + 0.7θ1 = 0 θk − 1.2θk−1 + 0.7θk−2 = 0
θ3 = 0.27,
θk = 1.2θk−1 − 0.7θk−2.
(School of Economics) Applied Econometrics for Macro and Finance
Example: MA Representation of an ARMA(2, 1)
0 10 20 30 40
(School of Economics) Applied Econometrics for Macro and Finance Week 3 13 / 33
Forecasting with an AR(1)
Supposewehaveasampleofobservationsy1,…,yT andwishtoforecastyT+h. The optimal predictor of a future value yT+h is the conditional expectation
y T + h ≡ E ( y T + h | y 1 , . . . , y T ) .
If y1, . . . , yT +h are generated by an AR(1) process:
E(yT+h|y1,…,yT)=E(a0 +a1yT+h−1 +εT+h| · ) =a0+a1E(yT+h−1| · )+E(εT+h| · )
=a0+a1(a0+a1E(yT+h−2| · )+E(εT+h−1| · )) .
=a 1+a +a2+···+ah−1+ahy , 01111T
y T + h = 1 − a h1 a 0 + a h1 y T . 1−a1
(School of Economics) Applied Econometrics for Macro and Finance Week 3 14 / 33
Forecasting with an AR(1)
Point forecasts on their own are difficult to interpret. To make them useful, we need to quantify uncertainty.
The primary source of uncertainty is due to an unknown future, i.e. future errors εT+1,…,εT+h.
This uncertainty can be quantified using the forecast error variance Var(yT +h − yT +h). Given an AR(1), the forecast error variance is:
σ2 ≡Var(yT+h−yT+h)= 1 σε2.
y,T +h 1 − a21
Question: What happens to point forecasts and forecast error variance when h is small? What happens as h −→ ∞?
(School of Economics) Applied Econometrics for Macro and Finance Week 3 15 / 33
Forecasting with an ARMA(p, q)
IF y1,…,yT+h are generated by an ARMA(p,q) process, yT+h ≡ E(yT+h |y1,…,yT) is generally non-linear and depends on the distribution of ε1, . . . , εT +h.
If the ARMA(p, q) is invertible, a good approximation yT +h ≈ yT +h is obtained from the pure AR representation: φ(L)yt = a0/b(1) + εt.
Assuming y0 = · · · = y−∞ = 0, point forecasts are obtained recursively: yT+1 =a0/b(1)+φ1yT +···+φTy1,
yT+2 =a0/b(1)+φ1yT+1 +φ2yT +···+φT+1y1, .
yT+h =a0/b(1)+φ1yT+h−1 +···+φh−1yT+1 +φhyT +···+φT+h−1y1.
(School of Economics) Applied Econometrics for Macro and Finance Week 3 16 / 33
Forecasting with an ARMA(p, q)
The forecast error variance is obtained from the first h − 1 terms of θ(L) = b(L)/a(L): σ 2 ≡ V a r ( y T + h − y T + h ) = ( 1 + θ 12 + · · · + θ h2 − 1 ) σ ε2 .
Note that the ARMA(p, q) does not need to be stable to compute the forecast error variance
for a finite h.
In the limit as h −→ ∞, the point forecast and forecast error variance exist only if the
ARMA(p, q) is stable, in which case:
lim yT+h = lim yT+h =α0/(1−α1 −···−αp),
lim σ2 = 1+θ2 σ2.
(School of Economics)
j=1 Applied Econometrics for Macro and Finance
∞ h−→∞y,T+h jε
Forecasting with an ARMA(p, q)
The point forecasts and forecast error variances computed from a general ARMA model yield predictive intervals, e.g. yT +h ± zασy,h.
zα is a constant obtained from the distribution of εt given a probability level α.
The general ARMA model, however, contains unknown elements, which are: p, q, a0, . . . , ap,
b1,…,bq and σε2.
A realised ARMA model is obtained by assigning values to all unknown elements. Predictive intervals are used to form predictive inference from realised ARMA(p, q) models.
(School of Economics) Applied Econometrics for Macro and Finance Week 3 18 / 33
Realisations of an ARMA Model
In practice, a realisation of an ARMA model is obtained through:
1 model specification;
2 model estimation.
ARMA model specification consists of:
setting lag orders p and q;
setting restrictions on a0,…,ap, b1,…,bq and σε2.
ARMA model estimation consists of computing values for a0, . . . , ap, b1, . . . , bq and σε2 from a sample y1, . . . , yT , given p, q and subject to restrictions.
(School of Economics) Applied Econometrics for Macro and Finance Week 3 19 / 33
Specification and Estimation Uncertainty
Unknown elements in an ARMA are additional sources of uncertainty: specification uncertainty and estimation uncertainty.
this uncertainty needs to be accounted for in predictive inference.
Estimation uncertainty leads to many possible realisations of a specified ARMA(p, q).
Predictive intervals can be systematically adjusted to account for this.
There exist many methods for quantifying uncertainty due to model specification. The simplest approach is:
identify an adequate set of specified ARMA(p, q) models;
estimate each specified ARMA in the set and compute predictive intervals; combine the predictive intervals to form predictive inference.
(School of Economics) Applied Econometrics for Macro and Finance Week 3 20 / 33
Set of Adequate ARMA(p, q) Models
The general class of ARMA(p, q) models is useful for forecasting because it is:
flexible: direct connection to the Wold representation;
simple: possible to analyse many theoretical features;
practical: computing predictive intervals from a realised ARMA is straightforward.
The flexibility of ARMAs translates to variability in predictive intervals.
The goal is to reduce the class of ARMAs to an adequate set of specified ARMAs. set of models too wide ⇒ more uncertainty and less accurate inference;
set of models too narrow ⇒ more bias and less accurate inference;
To strike the optimal balance between flexibility and variability, we identify an adequate set of specified ARMA(p, q) models by using:
data, rigorous statistical methods, assumptions, common sense.
(School of Economics) Applied Econometrics for Macro and Finance Week 3 21 / 33
Estimation of ARMA(p, q) Models
Identifying an adequate set of specified ARMA(p, q) models is an iterative process that
involves estimation.
Given orders p, q and restrictions, the parameters a0, . . . , a1, b1, . . . , bq and σε2 can be
estimated using the sample y1, . . . , yT .
If q = 0, and no restrictions are imposed, the resulting pure AR(p) model can be estimated
using OLS; in general, Maximum Likelihood (MLE) is typically used.
MLE chooses parameters a0, . . . , ap, b1, . . . , bq and σε2 to maximize the likelihood subject to
restrictions (if any).
A likelihood is the joint density of the observed data (e.g. y1, . . . , yT ) as a function of the model parameters.
Assuming a distribution for the errors implies a likelihood for data.
(School of Economics) Applied Econometrics for Macro and Finance Week 3 22 / 33
Estimation of ARMA(p, q) Models
Estimation maps data to parameter values called estimates.
Estimators a0,…,ap, b1,…,bq and σε are functions of y1,…,yT.
For a given sample y1, . . . , yT , they produce exactly one value for each parameter. Different samples of size T result in different estimated parameter values.
Estimation uncertainty can be quantified by Var(a0,…,ap,b1,…,bq,σε), which is used to
construct parameter confidence sets.
Confidence sets generate a set of realised ARMA models for a given specification.
Confidence sets can be incorporated into predictive intervals to capture estimation uncertainty (details of the methodology are beyond the scope of this course).
Estimators are designed to converge to specific values as T −→ ∞ so that
2 Var(a0,…,ap,b1,…,bq,σε) −→ 0.
(School of Economics) Applied Econometrics for Macro and Finance Week 3 23 / 33
Estimation of ARMA(p, q) Models
Estimators can be constructed to impose parameter restrictions.
For example, we can impose stability by restricting a1, . . . , ap to satisfy 1−a1z−···−apzp ̸=0forall|z|≤1.
In the AR(1) case, this is simply −1 < a1 < 1;
for more general AR(p) processes, the restrictions are more complicated.
Similarly, we can impose invertibility by restricting b1, . . . , bp to satisfy 1−b1z−···−bqzq ̸=0forall|z|≤1.
Alternatively, we can impose restrictions such as a1 = 1 or bj = 0 for some j < q.
Restrictions often make estimation more complicated; they can sometimes be “tested” instead of imposing in estimation.
(School of Economics) Applied Econometrics for Macro and Finance Week 3 24 / 33
Selection of Lag Orders and Restrictions
Modern statistical software can estimate ARMA(p, q) models very quickly!
It is feasible to estimate a large set of ARMA(p, q) models in a short amount of time.
For example, all combinations of p = 0,...,10 and q = 0,...,10 requires estimating 121 models, but this is typically not a problem.
We can start with a large set of candidate models and use estimation results to reduce the set; the guiding principles are:
fit—how closely a specified ARMA model matches the sample; parsimony—all else equal, less parameters is better; assumptions—mean-independence of residuals is crucial.
(School of Economics) Applied Econometrics for Macro and Finance Week 3 25 / 33
Criteria for Model Selection
Once a set of models is estimated, we can use information criteria to compare the models in a more formal way and reduce the set.
Two commonly used information criteria are: Akaike’s Information Criterion (AIC) and Bayesian Information Criterion (BIC, or sometimes SC, SBC, SBIC).
All Information criteria quantify a penalty for lack of fit and over-parameterizations. When comparing models using information criteria, lower is better.
The trade off is fit versus parsimony.
Both AIC and BIC are likelihood based:
2 p+q+1 2 p+q+1
AIC=lnσε +2 T , BIC=lnσε +lnT T . (School of Economics) Applied Econometrics for Macro and Finance Week 3
Using AIC and BIC in Practice
AIC and BIC both sum the log of the estimated residual variance (i.e. measure of fit) and an estimate of the number of free parameters.
They differ in the way the number of free parameters are estimated.
AIC and BIC are useful to compare two or more models: they do not provide an absolute
assessment of a given model’s suitability.
AIC and BIC can be used to compare any models where εt retains the same meaning.
For example, ARMA models of different orders p, q and with different parameter restrictions, as long they are modelling the same process {yt}.
AIC and BIC often disagree on the ranking of alternative models, and both are subject to estimation uncertainty.
Comparing models with AIC and BIC provides useful but partial information.
(School of Economics) Applied Econometrics for Macro and Finance Week 3 27 / 33
Analysing the Residuals
The residual εt in an ARMA(p, q) model is assumed to be mean-independent.
An estimator of the residual εt is obtained by imposing invertibility on b1, . . . , bq.
Computeφ(L)=a(L)/b(L)andε =y −φ y −···−φ y −a /b(1).
t t 1 t−1 t−1 1 0 Mean-independence implies the process {εt} has zero autocorrelation: corr(εt, εs) = 0 for all
t ̸= s; such a process is called white noise.
If the residuals are not white noise, then they are not mean-independent. Note: residuals could be white noise, but not mean-independent.
Givenestimatesε,...,ε ,wecancomputesampleautocorrelationsfork=1,...,T−2: 1T
sample autocorrelations always exist!
Consequently, we obtain the sample ACF (SACF) and sample PACF (SPACF). The SACF and SPACF provide an indication whether residuals are white noise.
(School of Economics) Applied Econometrics for Macro and Finance Week 3 28 / 33
The Q-Statistic and Ljung-Box Test
Many software packages use the SACF and SPACF of residuals to test white noise.
A commonly reported test is based on the Ljung-Box (1978) Q-statistic.
It is a joint test that the first K autocorrelations (ρk) are not significant. H0 :ρ1 =ρ2 =···=ρK =0versusH1 :someρk ̸=0.
The test statistic is computed directly from the SACF as
QK =T(T+2)
where rk is the sample autocorrelation at lag k, and K is the number of autocorrelation lags
being tested.
(School of Economics) Applied Econometrics for Macro and Finance Week 3 29 / 33
Sampling Distribution of the Q-statistic
Under certain assumptions, the sampling distribution of QK is χ2ν, where:
ν = K if QK computed directly from data;
ν = K−p − q if QK is computed from estimated ARMA(p, q) residuals.
The crucial assumptions required to justify an asymptotic χ2ν for QK computed from estimated ARMA(p, q) residuals is that the process under the null hypothesis:
is stable and invertible;
has homoscedastic residuals.
Other tests, such as the Durbin-Watson and Breusch-Godfrey (also called ) among many others, require alternative assumptions.
Different tests have different properties in different settings.
Tests involving estimated ARMA(p, q) residuals must be carried out and interpreted with
(School of Economics) Applied Econometrics for Macro and Finance Week 3 30 / 33
Analysing the Residuals in Practice
Software typically computes ε , . . . , ε along with SACF, SPACF, Q-statistics and p-values for 1T
the Ljung-Box. In this case, p-values are based on QK ∼ χ2K .
(School of Economics) Applied Econometrics for Macro and Finance Week 3 31 / 33
Inference About the Residuals in Practice
For q ≥ 1, estimated residuals ε are more accurate towards the end of the sample t
(t = T, T − 1, . . . ) and less accurate towards the beginning (t = 1, 2, . . . ). Accuracy depends on the assum
程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com