QBUS 6840 Lecture 9 Seasonal ARIMA Models Model Combination
QBUS 6840 Lecture 9
Copyright By PowCoder代写 加微信 powcoder
Seasonal ARIMA Models
Model Combination
The University of School
Objectives
Understand the seasonality and the ways to deal with it
Be able to distinguish seasonal ARIMA from normal ARIMA
Be able to identify seasonal patterns from ACF/PACF
Understand the seasonal Box-Jenkins models
Be able to convert between models and their B-forms
Understand the concept of model combination
Know some basic combination methods
Review of ARMA(p, q) and ARIMA(p, d , q) Processes
ARMA(p, q) Formulation with backshift operators(
ARIMA(p, d , q) Formulation with backshift operators(
(1− B)dYt = c +
Let Zt = (1− B)dYt , then Zt is the d-order differencing of
Yt . Hence ARMA(p, q) of Zt is the ARMA(p, d , q) of Yt
These are nonseasonal models
Review of ARMA(p, q) and ARIMA(p, d , q) Processes
ARMA(p, q) Formulation with backshift operators(
ARIMA(p, d , q) Formulation with backshift operators(
(1− B)dYt = c +
Let Zt = (1− B)dYt , then Zt is the d-order differencing of
Yt . Hence ARMA(p, q) of Zt is the ARMA(p, d , q) of Yt
These are nonseasonal models
Procedure to Estimate ARMA(p, q)/ARIMA(p, d , q)
processes: Lecture08 Example04.py
1 For the given time series {Yt}, check its stationarity by looking at
its Sample ACF and Sample PACF.
2 If ACF does not die down quickly, which means the given time series
{Yt} is nonstationary, we seek for a transformation, e.g., log
transformation {Zt = log(Yt)}, or the first order difference
{Zt = Yt − Yt−1}, or even the difference of log time series, or the
difference of the first order difference, so that the transformed time
series is stationary by checking its Sample ACF
3 When both Sample ACF and Sample PACF die down quickly, check
the orders at which ACF or PACF die down. The order of ACF will
be the lag q of the ARIMA and the order of PACF will be the lag p
of the ARIMA, and the order of difference will be d .
4 Estimate the identified ARIMA(p, d , q), or ARMA(p, q) (if we did
not do any difference transformation)
5 Make forecast with estimated ARIMA(p, d , q), or ARMA(p, q)
Seasonal ARIMA models
Seasonal ARIMA models
Variance stabilising transform
This is the Log transformed data.
Seasonal ARIMA models
ACF and PACF for the log visitors series
Seasonal ARIMA models
First differenced log visitors series
Seasonal ARIMA models
ACF and PACF for the first differenced log visitors series
Seasonal ARIMA models
Seasonal differencing
We can use seasonal differencing to remove the nonstationarity
caused by the seasonality:
yt − yt−12
Seasonal ARIMA models
Seasonally differenced log visitors series
Take the first seasonal difference
Seasonal ARIMA models
ACF and PACF for the seasonally differenced log visitors series
ACF suggests that the transformed series is stationary, but there is
clearly a seasonal effect on both ACF and PACF. We can use
seasonal ARMA to model this transformed time series
Seasonal AR(1)m model
Seasonal AR(1)m model takes the following form
Yt = c + Φ1Yt−m + εt
with m the seasonal frequency, e.g., m = 12 for monthly
In the form of B operator
(1− Φ1Bm)Yt = c + εt
Compared to the non-seasonal AR(1)
Yt = c + φ1Yt−1 + εt
Note the use of the notations Φ1 and φ1
ACF of AR(1)m
Φi1, if k = i ×m, i = 0, 1, 2, …
0, otherwise
AR(1)m: ACF and PACF
drawing and guess what is AR(2)m
Seasonal MA(1)m model
Seasonal MA(1)m model takes the following form
Yt = c + Θ1εt−m + εt
with m the seasonal frequency, e.g., m = 12 for monthly
In the form of B operator
Yt = c + (1 + Θ1B
Compared to the non-seasonal MA(1)
Yt = c + θ1εt−1 + εt
Note the use of the notations Θ1 and θ1
ACF of MA(1)m
, if k = m
0, k ≥ 1 and k 6= m
MA(1)m: ACF and PACF
drawing and guess what is MA(2)m
Seasonal ARMA(P ,Q)m model
The general form of ARMA(P,Q)m model(
where the error terms εts are white noise with mean zero and
variance σ2.
Conditions for stationarity and invertibility are the same as for
ARMA. E.g., AR(1)m is stationary if |Φ1| < 1
For AR(P)m (which is ARMA(P, 0)m): ACF dies down at lags
im, i = 0, 1, 2, ... and PACF cuts off after lag Pm.
For MA(Q)m (which is ARMA(0,Q)m): ACF cuts off after lag
Qm and PACF dies down at lags im, i = 0, 1, 2, ....
For ARMA(P,Q)m: both ACF and PACF die down at lags
im, i = 0, 1, 2, ....
ARMA(P ,Q)m: ACF and PACF
Mixed Seasonal ARMA(p, q)(P ,Q)m
In practice, many time series can be modeled well by a mixed
seasonal ARMA model
Yt = c +
where the error terms εts are white noise with mean zero and
variance σ2.
For example, an ARMA(0, 1)(1, 0)12 model is a combination
of a seasonal AR(1)12 and a non-seasonal MA(1)
Yt − Φ1Yt−12 = c + εt + θ1εt−1
The behavior of ACF and PACF is a combination of behavior
of the seasonal and nonseasonal parts of the model.
Mixed Seasonal ARMA(p, q)(P ,Q)m: Example
We can conclude that the seasonal part is MA(1)12. The orders p
and q of the non-seasonal part ARMA(p, q) are not clear - model
selection criteria are needed
Seasonal ARIMA(p, d , q)(P ,D,Q)m Model
Seasonal ARMA and mixed seasonal ARMA require
stationarity
For non-stationary time series Yt , by taking d-order difference
d(Yt) = (1− B)d(Yt)
and D-order seasonal difference
m(Yt) = (1− B
we can arrive at a transformed time series Zt which is
stationary
By combining difference and seasonal difference with mixed
seasonal ARMA, we have a very general seasonal
autoregressive integrated moving average (Seasonal ARIMA)
model, also called Seasonal Box-Jenkins models.
Seasonal Box-Jenkins models
ARIMA(p, d , q)(P,D,Q)m models
where m = number of seasonal period (e.g. m = 12).
(1 − φ1B − φ2B
2 − · · · − φpB
2m − · · · − ΦPB
)(1 − B)d (1 − Bm)DYt
=c + (1 + θ1B + θ2B
+ · · · + θqB
+ · · · + ΘQB
Seasonal Box-Jenkins models: Example
The above is an ARIMA(1, 1, 1)(1, 1, 1)4 model (with c = 0)
Seasonal Box-Jenkins models
ARIMA(1, 0, 0)(0, 1, 1)12 for monthly data
ARIMA(1, 0, 0)(0, 1, 1)12 for monthly data:
(1− φ1B)(1− B12)Yt = c + (1 + Θ1B12)εt
This is equivalent to
Yt − Yt−12 = c + φ1(Yt−1 − Yt−13) + εt + Θ1εt−12
Seasonal Box-Jenkins models
ARIMA(1, 0, 0)(1, 0, 0)12 models
(1− φ1B)(1− Φ1B12)Yt = c + εt
Or write it out
Yt = c + φ1Yt−1 + Φ1Yt−12 − φ1Φ1Yt−13 + εt
This is because informally
(1− φ1B)(1− Φ1B12) = 1− φ1B − Φ1B12 + φ1Φ1B13
(1− φ1B)(1− Φ1B12)Yt = Yt − φ1Yt−1 − Φ1Yt−12 + φ1Φ1Yt−13
Seasonal Box-Jenkins models
ARIMA(1, 0, 0)(1, 0, 0)12 models
(1− φ1B)(1− Φ1B12)Yt = c + εt
Or write it out
Yt = c + φ1Yt−1 + Φ1Yt−12 − φ1Φ1Yt−13 + εt
This is because informally
(1− φ1B)(1− Φ1B12) = 1− φ1B − Φ1B12 + φ1Φ1B13
(1− φ1B)(1− Φ1B12)Yt = Yt − φ1Yt−1 − Φ1Yt−12 + φ1Φ1Yt−13
This is how
Seasonal Box-Jenkins models
ARIMA(p, d , q)(P,D,Q)m models
ARIMA(1, 1, 1)(1, 1, 0)12 model:
(1− φ1B)(1− Φ1B12)(1− B)(1− B12)Yt = c + (1 + θ1B)εt
First denote Zt = (1−B)(1−B12)Yt . For this new time series Zt ,
the model is
(1− φ1B)(1− Φ1B12)Zt = c + (1 + θ1B)εt
(1− φ1B − Φ1B12 + φ1Φ1B13)Zt = c + (1 + θ1B)εt
Zt = φ1BZt + Φ1B
12Zt − φ1Φ1B13Zt + c + εt + θ1Bεt
Zt = φ1Zt−1 + Φ1Zt−12 − φ1Φ1Zt−13 + c + εt + θ1εt−1
Seasonal Box-Jenkins models
ARIMA(p, d , q)(P,D,Q)m models
ARIMA(1, 1, 1)(1, 1, 0)12 model:
(1− φ1B)(1− Φ1B12)(1− B)(1− B12)Yt = c + (1 + θ1B)εt
First denote Zt = (1−B)(1−B12)Yt . For this new time series Zt ,
the model is
(1− φ1B)(1− Φ1B12)Zt = c + (1 + θ1B)εt
(1− φ1B − Φ1B12 + φ1Φ1B13)Zt = c + (1 + θ1B)εt
Zt = φ1BZt + Φ1B
12Zt − φ1Φ1B13Zt + c + εt + θ1Bεt
Zt = φ1Zt−1 + Φ1Zt−12 − φ1Φ1Zt−13 + c + εt + θ1εt−1
Seasonal Box-Jenkins models
ARIMA(p, d , q)(P,D,Q)m models
ARIMA(1, 1, 1)(1, 1, 0)12 model:
(1− φ1B)(1− Φ1B12)(1− B)(1− B12)Yt = c + (1 + θ1B)εt
First denote Zt = (1−B)(1−B12)Yt . For this new time series Zt ,
the model is
(1− φ1B)(1− Φ1B12)Zt = c + (1 + θ1B)εt
(1− φ1B − Φ1B12 + φ1Φ1B13)Zt = c + (1 + θ1B)εt
Zt = φ1BZt + Φ1B
12Zt − φ1Φ1B13Zt + c + εt + θ1Bεt
Zt = φ1Zt−1 + Φ1Zt−12 − φ1Φ1Zt−13 + c + εt + θ1εt−1
Seasonal Box-Jenkins models
ARIMA(p, d , q)(P,D,Q)m models
ARIMA(1, 1, 1)(1, 1, 0)12 model:
(1− φ1B)(1− Φ1B12)(1− B)(1− B12)Yt = c + (1 + θ1B)εt
First denote Zt = (1−B)(1−B12)Yt . For this new time series Zt ,
the model is
(1− φ1B)(1− Φ1B12)Zt = c + (1 + θ1B)εt
(1− φ1B − Φ1B12 + φ1Φ1B13)Zt = c + (1 + θ1B)εt
Zt = φ1BZt + Φ1B
12Zt − φ1Φ1B13Zt + c + εt + θ1Bεt
Zt = φ1Zt−1 + Φ1Zt−12 − φ1Φ1Zt−13 + c + εt + θ1εt−1
Seasonal Box-Jenkins models
ARIMA(p, d , q)(P,D,Q)m models
ARIMA(1, 1, 1)(1, 1, 0)12 model:
(1− φ1B)(1− Φ1B12)(1− B)(1− B12)Yt = c + (1 + θ1B)εt
Finally from
Zt = φ1Zt−1 + Φ1Zt−12 − φ1Φ1Zt−13 + c + εt + θ1εt−1
Zt =(1− B)(1− B12)Yt = Yt − Yt−1 − Yt−12 + Yt−13
=(Yt − Yt−1)− (Yt−12 − Yt−13)
(Yt − Yt−1)− (Yt−12 − Yt−13) =c + φ1 [(Yt−1 − Yt−2)− (Yt−13 − Yt−14)]
+ Φ1 [(Yt−12 − Yt−13)− (Yt−24 − Yt−25)]
− φ1Φ1 [(Yt−13 − Yt−14)− (Yt−25 − Yt−26)]
+ εt + θ1εt−1
Seasonal Box-Jenkins models
ARIMA(p, d , q)(P,D,Q)m models
ARIMA(1, 1, 1)(1, 1, 0)12 model:
(1− φ1B)(1− Φ1B12)(1− B)(1− B12)Yt = c + (1 + θ1B)εt
Finally from
Zt = φ1Zt−1 + Φ1Zt−12 − φ1Φ1Zt−13 + c + εt + θ1εt−1
Zt =(1− B)(1− B12)Yt = Yt − Yt−1 − Yt−12 + Yt−13
=(Yt − Yt−1)− (Yt−12 − Yt−13)
(Yt − Yt−1)− (Yt−12 − Yt−13) =c + φ1 [(Yt−1 − Yt−2)− (Yt−13 − Yt−14)]
+ Φ1 [(Yt−12 − Yt−13)− (Yt−24 − Yt−25)]
− φ1Φ1 [(Yt−13 − Yt−14)− (Yt−25 − Yt−26)]
+ εt + θ1εt−1
Seasonal Box-Jenkins models
ARIMA(p, d , q)(P,D,Q)m models
ARIMA(1, 1, 1)(1, 1, 0)12 model:
(1− φ1B)(1− Φ1B12)(1− B)(1− B12)Yt = c + (1 + θ1B)εt
Finally from
Zt = φ1Zt−1 + Φ1Zt−12 − φ1Φ1Zt−13 + c + εt + θ1εt−1
Zt =(1− B)(1− B12)Yt = Yt − Yt−1 − Yt−12 + Yt−13
=(Yt − Yt−1) − (Yt−12 − Yt−13)
(Yt − Yt−1) − (Yt−12 − Yt−13) =c + φ1 [(Yt−1 − Yt−2) − (Yt−13 − Yt−14)]
+ Φ1 [(Yt−12 − Yt−13)− (Yt−24 − Yt−25)]
− φ1Φ1 [(Yt−13 − Yt−14)− (Yt−25 − Yt−26)]
+ εt + θ1εt−1
Seasonal ARIMA models
First and seasonal differencing
In our example for the log visitors series, we saw that seasonally
differencing is not enough to make the series stationary. We can
then consider the transform:
(1− B12)(1− B)Yt = (Yt − Yt−1)− (Yt−12 − Yt−13)
(Yt − Yt−1)− (Yt−12 − Yt−13) = (Yt − Yt−12)− (Yt−1 − Yt−13)
= (1− B12)Yt − (1− B12)Yt−1
denote Zt = (1− B12)Yt : = Zt − Zt−1 = (1− B)Zt
= (1− B)(1− B12)Yt
(1− B12)(1− B)Yt = (1− B)(1− B12)Yt
Seasonal ARIMA models
First and seasonal differencing
In our example for the log visitors series, we saw that seasonally
differencing is not enough to make the series stationary. We can
then consider the transform:
(1− B12)(1− B)Yt = (Yt − Yt−1)− (Yt−12 − Yt−13)
(Yt − Yt−1)− (Yt−12 − Yt−13) = (Yt − Yt−12)− (Yt−1 − Yt−13)
= (1− B12)Yt − (1− B12)Yt−1
denote Zt = (1− B12)Yt : = Zt − Zt−1 = (1− B)Zt
= (1− B)(1− B12)Yt
(1− B12)(1− B)Yt = (1− B)(1− B12)Yt
Seasonal ARIMA models
First and seasonal differencing
In our example for the log visitors series, we saw that seasonally
differencing is not enough to make the series stationary. We can
then consider the transform:
(1− B12)(1− B)Yt = (Yt − Yt−1)− (Yt−12 − Yt−13)
(Yt − Yt−1)− (Yt−12 − Yt−13) = (Yt − Yt−12)− (Yt−1 − Yt−13)
= (1− B12)Yt − (1− B12)Yt−1
denote Zt = (1− B12)Yt : = Zt − Zt−1 = (1− B)Zt
= (1− B)(1− B12)Yt
(1− B12)(1− B)Yt = (1− B)(1− B12)Yt
Seasonal ARIMA models
First and seasonal differencing
In our example for the log visitors series, we saw that seasonally
differencing is not enough to make the series stationary. We can
then consider the transform:
(1− B12)(1− B)Yt = (Yt − Yt−1)− (Yt−12 − Yt−13)
(Yt − Yt−1)− (Yt−12 − Yt−13) = (Yt − Yt−12)− (Yt−1 − Yt−13)
= (1− B12)Yt − (1− B12)Yt−1
denote Zt = (1− B12)Yt : = Zt − Zt−1 = (1− B)Zt
= (1− B)(1− B12)Yt
(1− B12)(1− B)Yt = (1− B)(1− B12)Yt
Seasonal ARIMA models
First and seasonal differencing
In our example for the log visitors series, we saw that seasonally
differencing is not enough to make the series stationary. We can
then consider the transform:
(1− B12)(1− B)Yt = (Yt − Yt−1)− (Yt−12 − Yt−13)
(Yt − Yt−1)− (Yt−12 − Yt−13) = (Yt − Yt−12)− (Yt−1 − Yt−13)
= (1− B12)Yt − (1− B12)Yt−1
denote Zt = (1− B12)Yt : = Zt − Zt−1 = (1− B)Zt
= (1− B)(1− B12)Yt
(1− B12)(1− B)Yt = (1− B)(1− B12)Yt
Seasonal ARIMA models
First and seasonal differencing
In our example for the log visitors series, we saw that seasonally
differencing is not enough to make the series stationary. We can
then consider the transform:
(1− B12)(1− B)Yt = (Yt − Yt−1)− (Yt−12 − Yt−13)
(Yt − Yt−1)− (Yt−12 − Yt−13) = (Yt − Yt−12)− (Yt−1 − Yt−13)
= (1− B12)Yt − (1− B12)Yt−1
denote Zt = (1− B12)Yt : = Zt − Zt−1 = (1− B)Zt
= (1− B)(1− B12)Yt
(1− B12)(1− B)Yt = (1− B)(1− B12)Yt
Seasonal Box-Jenkins models
ARIMA(p, d , q)(P,D,Q)m models
ARIMA(0, 0, 0)(P, 0, 0)
Sample autocorrelations die down for lags m, 2m, 3m, etc.
Sample partial autocorrelations cut off at lag Pm.
ARIMA(0, 0, 0)(0, 0,Q)
Sample autocorrelations cuts off at lag Qm.
Sample partial autocorrelations die down for lags m, 2m, 3m,
ARIMA(0, 0, 0)(0, 1, 0)
Sample autocorrelations and partial autocorrelations die down
very slowly for lags m, 2m, 3m, etc.
Seasonal ARIMA models
Lecture09 Example01.py
Carefully read the scripts in Lecture09 Example01.py
The concluded model is
(1−φ1B−φ2B2−φ3B3−φ4B4−φ5B5)(1−B12)Zt = εt +Θ1B12εt
(1− φ1B − φ3B3 − φ5B5)(1− B12)Zt = εt + Θ1B12εt
where Zt is the quartic root data, i.e., Zt = Y
Seasonal ARIMA models
Lecture09 Example01.py
Carefully read the scripts in Lecture09 Example01.py
The concluded model is
(1−φ1B −φ2B2 −φ3B3 −φ4B4 −φ5B5)(1−B12)Zt = εt +Θ1B12εt
(1 − φ1B − φ3B3 − φ5B5)(1 − B12)Zt = εt + Θ1B12εt
where Zt is the quartic root data, i.e., Zt = Yt
Seasonal ARIMA models
First and seasonal differencing
Seasonal ARIMA models
First and seasonal differencing
Seasonal ARIMA models
First and seasonally differenced log visitors: estimation
ARIMA(2, 1, 2)(0, 1, 1)12 model:
(1−φ1B−φ2B2)(1−B)(1−B12)Yt = c+(1+θ1B+θ2B2)(1+Θ1B12)εt
Estimated coefficients (using R):
ar1 ar2 ma1 ma2 sma1
-0.7817 -0.3154 -0.0300 -0.4007 -0.7471
s.e. 0.2212 0.1227 0.2213 0.1909 0.1073
log likelihood=178.99, AIC=-345.97, AICc=-345.08, BIC=-330.28.
Seasonal ARIMA models
ARIMA(2, 1, 2)(0, 1, 1)12 model: residuals
Seasonal ARIMA models
ARIMA(2, 1, 2)(0, 1, 1)12 model: forecasts
Seasonal ARIMA models
Intercept terms induce permanent trends.
Only seasonally difference once.
Usually only either one seasonal AR or MA term is needed.
Seasonal AR terms are often used when the lag m sample
autocorrelation terms are positive.
Seasonal MA terms are often used when the lag m sample
autocorrelation terms are negative.
Forecasting combinations
Introduction
Classical reference:
Bates, J. M., and C. W. J. Granger (1969). The combination of
forecasts, Operational Research Quarterly, 20, 451–468.
They provide the following illustration:
Forecasting combinations
Introduction
It is possible to combine unbiased forecasts ŷ
T+1|T from
models i = 1, . . . ,m.
The models can be various ARIMA type of models or a set of
ARIMA models, HW exponential smoothing models and
regression models for example.
T+1|T , i = 1, . . . ,m could also be m expert forecasts.
Forecasting combinations
The forecasts can be combined as follows
ŷ cT+1|T =
The simplest way is to set wi =
, then you are using a
simple average.
Simple averages often work surprisingly well.
We often use convex combinations, that is 0 ≤ w ≤ 1.
The question is how to combine forecasts “optimally”?
Forecasting combinations
Variance reduction: example of two forecasts (optional)
If you have two unbiased forecasts ŷ
T+1|T and ŷ
T+1|T with the
corresponding variances σ21 and σ
2, then we can combine them
ŷ cT+1|T = wŷ
T+1|T + (1− w)ŷ
The variance of the combined forecast will be
2σ21 + (1− w)
2σ22 + 2ρwσ1(1− w)σ2.
It will have minimum at
σ22 − ρσ1σ2
2 − 2ρσ1σ2
where w is the “optimal” value. In case where ŷ
T+1|T and ŷ
are uncorrelated (ρ = 0), then w = σ22/(σ
2), which is no
greater than the smaller of the two individual variances.
Forecasting combinations
Empirical weights (optional)
Unfortunately we dont know the actual σ21 and σ
design a way to estimate them.
One-step ahead forecasts over a sample T + 1 observations.
Dividing the T + 1 observations into an initial estimation
(regression) subsample (e.g., from time 1 to t0) and a second
evaluation (prediction) subsample (from t0 + 1 to T + 1).
The first subsample enables you to estimate the parameters of
each model.
In the second subsample, the forecasting performance of each
model can be evaluated. Each model’s performance will differ
from period to period:
e1,t0+1, e1,t0+2, ..., e1,T
e2,t0+1, e2,t0+2, ..., e2,T
Forecasting combinations
Empirical weights (optional)
We are going to use the estimates (many possibilities)
e21,t ; for t1 = t0 + 1, ...,T
e22,t ; for t1 = t0 + 1, ...,T
In general, we can define The simplest version of such
adaptive weights is, for t1 = t0 + 1, ...,T ,
Forecasting combinations
Empirical weights (optional)
At the time T , we can have the simplest version of such
adaptive weights
Or updating as (a clever way)
w∗T+1 = αw
T + (1− α)
= αw∗T + (1− α)wT+1
Forecasting combinations
There is extensive empirical evidence in favour of combinations as
a forecasting strategy.
Forecasting combinations offer diversification gains that make
them very useful compared to relying on a single model, as we
have just seen.
There may be structural breaks in the data, making it
plausible that combining models with different levels
adaptability will lead to better results than relying on a single
Forecasting combinations
Even without structural breaks, individuals models may be
subject to misspecification bias: it is implausible that a single
model dominates all others at all time periods.
An additional argument for combining forecasts is that
predictions from different forecasters may have been
constructed under different loss functions.
Forecasting combinations
Estimation errors are a serious problem for obtaining the
combination weights. Simple averages are often better.
Similarly, nonstationarities can cause instabilities on the
程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com