代写代考 QBUS2820 Lecture 13 ARIMA models (II)

Moving average (MA) processes ARMA(p, q) and ARIMA(p, d, q) processes
QBUS2820 Lecture 13 ARIMA models (II)
Discipline of Business Analytics
The University of School

Moving average (MA) processes
ARMA(p, q) and ARIMA(p, d, q) processes
Moving average MA(q) processes
Yt =c+εt+θ1εt−1+θ2εt−2+…+θqεt−q,
where εt is i.i.d. with mean zero and variance σ2.
a weighted moving average of the past few forecast errors.
appropriate to model quantities yt, such as economic indicators, which are affected by random events that have both immediate and persistent effect on yt
sometimes, the εt are called random shocks: shocks caused by unpredictable events
MA(q) process

Moving average (MA) processes
ARMA(p, q) and ARIMA(p, d, q) processes
MA(q) process
MA(1) process
Yt = c + εt + θ1εt−1.
Unconditional mean:
E[Yt] = E[c + εt + θ1εt−1] = c + 0 + θ1 × 0 = c
Unconditional variance:
V(Yt) = V(c) + V(εt) + V(θ1εt−1) =0+σ2 +σ2θ12 =σ2(1+θ12)

Moving average (MA) processes
ARMA(p, q) and ARIMA(p, d, q) processes
MA(q) process
MA(1) process: Properties
Covariance:
Cov(Yt,Yt−1)=Cov(c+εt +θ1εt−1,c+εt−1+θ1εt−2) =Cov(c,c)+Cov(c,εt−1)+Cov(c,θ1εt−2)+Cov(εt,c)
+Cov(εt,εt−1)+Cov(εt,θ1εt−2)+Cov(θ1εt−1,c)
+ Cov(θ1εt−1, εt−1) + Cov(θ1εt−1, θ1εt−2) =θ1Cov(εt−1, εt−1) = θ1V(εt−1) = θ1σ2
Autocorrelation:
Cov(Yt,Yt−1) θ1σ2 θ1 ρ1:= ==2 V(Yt) V(Yt) 1+θ1

Moving average (MA) processes
ARMA(p, q) and ARIMA(p, d, q) processes
MA(q) process
MA(1) process: Properties
Cov(Yt,Yt−2) = 0(Why?), ρ2 = 0.
ρk = 0 for k > 1.
Conclusion: MA(1) process is stationary for every θ1, and its ACF plot cuts off after lag 1
Partial ACF:
ρkk=−θ1k(1−θ12), k≥1. 1 − θ2(k+1)
Partial ACF plot dies down exponentially when |θ1| < 1. Moving average (MA) processes ARMA(p, q) and ARIMA(p, d, q) processes MA(1) process: Forecasting E(Yt+1|y1:t) = c + θ1εt We use the forecast errors 􏰇ε1, ..., 􏰇εt from the previous periods to construct the next forecast at time t + 1 Let the forecast at time t is y􏰇 , and forecast error t ε􏰇=y−y􏰇=y−(c+θε􏰇 ) tttt 1t−1 Forecast of Yt+1 is and forecast error y􏰇 = c + θ ε􏰇 t+1 1t ε􏰇 =y −y􏰇 =y −(c+θ ε􏰇). t+1 t+1 t+1 t+1 1 t The variance of the forecast is V(Yt+1|y1:t ) = σ2. MA(q) process Moving average (MA) processes ARMA(p, q) and ARIMA(p, d, q) processes MA(q) process MA(1) process: Forecasting E(Yt+2|y1:t) = c + E(εt+2|y1:t) + θ1E(εt+1|y1:t) = c, so y􏰇 =c. t +2|t V(Yt+2|y1:t) = σ2(1 + θ12) In general, it’s easy to see that y􏰇 = c for h > 1 t+h|t
V(Yt+h|y1:t) = σ2(1 + θ12) for h > 1

Moving average (MA) processes
ARMA(p, q) and ARIMA(p, d, q) processes
MA(q) process
MA(1) process: Forecasting

Moving average (MA) processes
ARMA(p, q) and ARIMA(p, d, q) processes
MA(q) process
Backshift operators
We now introduce the Backshift operator, which is very useful for describing time series models
BYt = Yt−1
B2Yt = B(BYt) = B(Yt−1) = Yt−2
BkYt =Yt−k Particularly for a constant series {d}, we define

Moving average (MA) processes
ARMA(p, q) and ARIMA(p, d, q) processes
MA(q) process
Backshift operators
In context: AR(1)
Yt = c + φ1Yt−1 + εt where gives μ = E(Yt) = E(Yt−1) = c/(1 − φ1)
(1−φ1B)Yt =c+εt (1−φ1B)(Yt −μ)=εt
which comes from the fact c = (1 − φ1)μ = (1 − φ1B)μ, which is from Bd = d for any constant d.
Denote Zt = Yt − μ, then
(1−φ1B)Zt =εt =⇒Zt =φ1Zt−1 +εt

Moving average (MA) processes
ARMA(p, q) and ARIMA(p, d, q) processes
MA(q) process
Backshift operators
In context: MA(1)
Yt = c + εt + θ1εt−1 which gives μ = E(Yt) = c.
Yt =c+(1+θ1B)εt (Yt −μ) = (1+θ1B)εt

Moving average (MA) processes
ARMA(p, q) and ARIMA(p, d, q) processes
MA(q) process
Backshift operators
In context: AR(p)
Yt =c+φ1Yt−1+…+φpYt−p+εt
= c + φ1 B (Yt ) + . . . + φp B p (Yt ) + εt
(1−φ1B−φ2B2−…−φpBp)(Yt −μ)=εt where μ = c /(1 − φ1 − φ2 − · · · − φp ),
(1−􏰆φiBi)(Yt −μ)=εt

Moving average (MA) processes
ARMA(p, q) and ARIMA(p, d, q) processes
MA(q) process
Backshift operators
In context: MA(q)
Yt =c+θ1εt−1+…+θqεt−q+εt
= c + θ1 B (εt ) + . . . + θq B q (εt ) + εt
(Yt −μ)=(1+θ1B+θ2B2+…+θqBq)εt
(Yt −μ)=(1+􏰆θiBi)εt

Moving average (MA) processes
ARMA(p, q) and ARIMA(p, d, q) processes
Invertibility
MA(q) process
An MA(q) process is invertible when we can rewrite it as a linear combination of its past values (an AR(∞)) plus the contemporaneous error term.
If we want to find the value εt at a certain period and the process is invertible, we need to know the current and past values of Y . For a noninvertible representation we would need to use all future values of Y !
Convenient algorithms for estimating parameters and forecasting are only valid if we use an invertible representation.

Moving average (MA) processes
ARMA(p, q) and ARIMA(p, d, q) processes
MA(q) process
Example: AR(1) as MA(∞)
Yt = c + φ1Yt−1 + εt
=c(1+φ1)+φ21Yt−2 +φ1εt−1 +εt
=c(1+φ1 +φ21)+φ21Yt−3 +φ21εt−2 +φ1εt−1 +εt
=c(1+φ1 +…+φt−1)+φt1y0 +􏰆φi1εt−i +εt
Y t = 1 − φ + φ i1 ε t − i + ε t

Moving average (MA) processes
ARMA(p, q) and ARIMA(p, d, q) processes
Checking Stationarity of AR(p)*1
Consider the AR(p) process
Yt =φ1Yt−1+φ2Yt−2+···+φpYt−p+εt
Accordingly, define the characteristic equation 1−φ1z−φ2z2−···−φpzp =0
whose roots are called the characteristic roots. There are p such roots, although some of them may be equal. Conclusion: The AR(p) is stationary if all the roots satisfy |z| > 1.
For example, the AR(1) is Yt = φ1Yt−1 + εt. The characteristic equation is 1 − φ1z = 0 and its only root is z∗ = 1/φ1. |z∗| > 1 implies the AR(1) stationarity. This means |φ1| < 1. MA(q) process 1Technical slide, ignore if you wish Moving average (MA) processes ARMA(p, q) and ARIMA(p, d, q) processes Checking Invertibility of MA(q)*2 Consider the MA(q) process Yt =εt +θ1εt−1 +θ2εt−2 +···+θqεt−q Accordingly, define the characteristic equation 1+θ1z+θ2z2+···+θqzq =0 whose roots are called the characteristic roots. There are q such roots, although some of them may be equal. Conclusion: The MA(q) is invertible if all the roots satisfy |z| > 1.
For example, the MA(1) is Yt = εt + θ1εt−1. The characteristic equation is 1 + θ1z = 0 and its only root is z∗ = −1/θ1. |z∗| > 1 implies the MA(1) is invertible. This means |θ1| < 1. MA(q) process 2Technical slide, ignore if you wish Moving average (MA) processes ARMA(p, q) and ARIMA(p, d, q) processes ARMA(p, q) processes ARIMA(p, d, q) processes ARMA(p, q) processes Yt =c+φ1Yt−1+...+φpYt−p+θ1εt−1+...+θqεt−q+εt, where εt is i.i.d. with mean zero and variance σ2. Example: ARMA(0, 0) : (White Noise) Example: ARMA(1, 1) : Yt =c+φ1Yt−1+θ1εt−1+εt, Moving average (MA) processes ARMA(p, q) processes ARMA(p, q) and ARIMA(p, d, q) processes ARIMA(p, d, q) processes ARMA(p, q) processes: Properties 1−φ1 −...−φp ρk dies down. ρkk dies down. Moving average (MA) processes ARMA(p, q) and ARIMA(p, d, q) processes ARMA(p, q) processes ARIMA(p, d, q) processes ARMA(1,1): Forecasting Yt+1 =c+φ1Yt +θ1εt +εt+1, y􏰇 =E(Y |y,...,y)=c+φy+θε Var(Yt+1|y1,...,yt) = σ2. t+1t+11t 1t1t Moving average (MA) processes ARMA(p, q) and ARIMA(p, d, q) processes ARMA(p, q) processes ARIMA(p, d, q) processes ARMA(1,1): Forecasting Yt+2 = c + φ1Yt+1 + θ1εt+1 + εt+2 = c + φ1(c + φ1Yt + θ1εt + εt+1) + θ1εt+1 + εt+2 =c(1+φ1)+φ21Yt +φ1θ1εt +(φ1 +θ1)εt+1 +εt+2 y􏰇 =c(1+φ)+φ2y +φθε t+2 1 1t 11t Var(Yt+2|y1,...,yt)=σ2(1+(φ1 +θ1)2). Moving average (MA) processes ARMA(p, q) and ARIMA(p, d, q) processes ARMA(p, q) processes ARIMA(p, d, q) processes Stationary transforms Box and Jenkins advocate difference transforms to achieve stationarity, e.g ∆Yt =Yt −Yt−1 ∆2Yt = (Yt − Yt−1) − (Yt−1 − Yt−2) = Yt − 2Yt−1 + Yt−1 Moving average (MA) processes ARMA(p, q) and ARIMA(p, d, q) processes ARMA(p, q) processes ARIMA(p, d, q) processes Stationary transforms Example: S&P 500 index Moving average (MA) processes ARMA(p, q) and ARIMA(p, d, q) processes ARMA(p, q) processes ARIMA(p, d, q) processes Stationary transforms Example: S&P 500 index Moving average (MA) processes ARMA(p, q) and ARIMA(p, d, q) processes ARMA(p, q) processes ARIMA(p, d, q) processes Stationary transforms Example: S&P 500 index Taking the first difference: Moving average (MA) processes ARMA(p, q) and ARIMA(p, d, q) processes ARMA(p, q) processes ARIMA(p, d, q) processes Stationary transforms Example: S&P 500 index Autocorrelations for the differenced series: Moving average (MA) processes ARMA(p, q) processes ARMA(p, q) and ARIMA(p, d, q) processes ARIMA(p, d, q) processes Autoregressive Integrated Moving Average Models: ARIMA(p,d,q) Suppose we consider the d-order difference of the original time series {Yt }. Denote Zt = ∆d Yt An ARMA(p,q) model on {Zt} is called an ARIMA(p,d,q) model on {Yt} Moving average (MA) processes ARMA(p, q) and ARIMA(p, d, q) processes ARMA(p, q) processes ARIMA(p, d, q) processes Seasonally adjusted visitor arrivals in Australia Example of modelling process Moving average (MA) processes ARMA(p, q) and ARIMA(p, d, q) processes ARMA(p, q) processes ARIMA(p, d, q) processes Seasonally adjusted visitor arrivals in Australia Variance stabilising transform We first take the log of the series as a variance stabilising transformation: Moving average (MA) processes ARMA(p, q) and ARIMA(p, d, q) processes ARMA(p, q) processes ARIMA(p, d, q) processes Log seasonally adjusted visitor arrivals in Australia ACF and PACF for the log series Moving average (MA) processes ARMA(p, q) and ARIMA(p, d, q) processes ARMA(p, q) processes ARIMA(p, d, q) processes Log seasonally adjusted visitor arrivals in Australia Stationary transform We then take the first difference: Moving average (MA) processes ARMA(p, q) and ARIMA(p, d, q) processes ARMA(p, q) processes ARIMA(p, d, q) processes Log seasonally adjusted visitor arrivals in Australia Differenced series Autocorrelations for the differenced series: Moving average (MA) processes ARMA(p, q) processes ARMA(p, q) and ARIMA(p, d, q) processes ARIMA(p, d, q) processes Log seasonally adjusted visitor arrivals in Australia Tentative model identification The ACF of the differenced series cuts off after lag one. The PACF seems to die down. This suggests that the differenced series may be an MA(1) process. The original log series would then be an ARIMA(0, 1, 1) process. Yt −Yt−1 = c +εt +θ1εt−1 Moving average (MA) processes ARMA(p, q) and ARIMA(p, d, q) processes ARMA(p, q) processes ARIMA(p, d, q) processes Log seasonally adjusted visitor arrivals in Australia Forecasting Moving average (MA) processes ARMA(p, q) and ARIMA(p, d, q) processes ARMA(p, q) processes ARIMA(p, d, q) processes Log seasonally adjusted visitor arrivals in Australia Residual analysis Moving average (MA) processes ARMA(p, q) and ARIMA(p, d, q) processes ARMA(p, q) processes ARIMA(p, d, q) processes ARMA(p, q) processes: Formulation with backshift operators 1−􏰆φiBi Yt =c+ 1+􏰆θiBi εt, i=1 i=1 Moving average (MA) processes ARMA(p, q) and ARIMA(p, d, q) processes ARMA(p, q) processes ARIMA(p, d, q) processes ARIMA(p, d, q) processes: Formulation with backshift operators 1−􏰆φiBi (1−B)dYt =c+ 1+􏰆θiBi εt, i=1 i=1 Moving average (MA) processes ARMA(p, q) processes ARMA(p, q) and ARIMA(p, d, q) processes ARIMA(p, d, q) processes Procedure to Estimate ARMA(p, q)/ARIMA(p, d, q) processes 1 For the given time series {Yt}, check its stationarity by looking at its Sample ACF. 2 If ACF does not die down quickly, which means the given time series {Yt } is nonstationary, we seek for a transformation, e.g., log transformation {Zt = log(Yt)}, or the first order difference {Zt = Yt − Yt−1}, or even the difference of log time series, or the difference of the first order difference, so that the transformed time series is stationary by checking its Sample ACF 3 When both Sample ACF and Sample PACF die down quickly, check the orders at which ACF or PACF die down to indentify tentatively the lags q and q of the ARIMA, and the order of difference will be d. 4 Estimate the identified ARIMA(p, d, q), or ARMA(p, q) (if we did not do any difference transformation) 5 Make forecast with estimated ARIMA(p, d, q), or ARMA(p, q) model Moving average (MA) processes ARMA(p, q) and ARIMA(p, d, q) processes ARMA(p, q) processes ARIMA(p, d, q) processes ARIMA(p, d, q) processes: Order selection 􏰃p􏰄 􏰃q􏰄 1−􏰆φiBi (1−B)dYt =c+ 1+􏰆θiBi εt, How to choose p (the order of AR) and q (the order of MA) when the ACF and PACF do not give us a straightforward answer? Use model selection methods such as AIC/BIC. Moving average (MA) processes ARMA(p, q) processes ARMA(p, q) and ARIMA(p, d, q) processes ARIMA(p, d, q) processes Review questions What is stationarity? What is ACF and PACF? How can you check stationarity? What is an AR(p) process? What is an MA(q) process? What is an ARMA(p, q) process?

Related Posts