留学生考试辅导 QBUS 6840 Lecture 9 Seasonal ARIMA Models Model Combination

QBUS 6840 Lecture 9 Seasonal ARIMA Models Model Combination

QBUS 6840 Lecture 9

Copyright By PowCoder代写 加微信 powcoder

Seasonal ARIMA Models
Model Combination

The University of School

Objectives

Understand the seasonality and the ways to deal with it

Be able to distinguish seasonal ARIMA from normal ARIMA

Be able to identify seasonal patterns from ACF/PACF

Understand the seasonal Box-Jenkins models

Be able to convert between models and their B-forms

Understand the concept of model combination

Know some basic combination methods

Review of ARMA(p, q) and ARIMA(p, d , q) Processes

ARMA(p, q) Formulation with backshift operators(

ARIMA(p, d , q) Formulation with backshift operators(

(1− B)dYt = c +

Let Zt = (1− B)dYt , then Zt is the d-order differencing of
Yt . Hence ARMA(p, q) of Zt is the ARMA(p, d , q) of Yt

These are nonseasonal models

Review of ARMA(p, q) and ARIMA(p, d , q) Processes

ARMA(p, q) Formulation with backshift operators(

ARIMA(p, d , q) Formulation with backshift operators(

(1− B)dYt = c +

Let Zt = (1− B)dYt , then Zt is the d-order differencing of
Yt . Hence ARMA(p, q) of Zt is the ARMA(p, d , q) of Yt

These are nonseasonal models

Procedure to Estimate ARMA(p, q)/ARIMA(p, d , q)
processes: Lecture08 Example04.py

1 For the given time series {Yt}, check its stationarity by looking at
its Sample ACF and Sample PACF.

2 If ACF does not die down quickly, which means the given time series
{Yt} is nonstationary, we seek for a transformation, e.g., log
transformation {Zt = log(Yt)}, or the first order difference
{Zt = Yt − Yt−1}, or even the difference of log time series, or the
difference of the first order difference, so that the transformed time
series is stationary by checking its Sample ACF

3 When both Sample ACF and Sample PACF die down quickly, check
the orders at which ACF or PACF die down. The order of ACF will
be the lag q of the ARIMA and the order of PACF will be the lag p
of the ARIMA, and the order of difference will be d .

4 Estimate the identified ARIMA(p, d , q), or ARMA(p, q) (if we did
not do any difference transformation)

5 Make forecast with estimated ARIMA(p, d , q), or ARMA(p, q)

Seasonal ARIMA models

Seasonal ARIMA models
Variance stabilising transform

This is the Log transformed data.

Seasonal ARIMA models
ACF and PACF for the log visitors series

Seasonal ARIMA models
First differenced log visitors series

Seasonal ARIMA models
ACF and PACF for the first differenced log visitors series

Seasonal ARIMA models
Seasonal differencing

We can use seasonal differencing to remove the nonstationarity
caused by the seasonality:

yt − yt−12

Seasonal ARIMA models
Seasonally differenced log visitors series

Take the first seasonal difference

Seasonal ARIMA models
ACF and PACF for the seasonally differenced log visitors series

ACF suggests that the transformed series is stationary, but there is
clearly a seasonal effect on both ACF and PACF. We can use
seasonal ARMA to model this transformed time series

Seasonal AR(1)m model

Seasonal AR(1)m model takes the following form

Yt = c + Φ1Yt−m + εt

with m the seasonal frequency, e.g., m = 12 for monthly

In the form of B operator

(1− Φ1Bm)Yt = c + εt

Compared to the non-seasonal AR(1)

Yt = c + φ1Yt−1 + εt

Note the use of the notations Φ1 and φ1
ACF of AR(1)m

Φi1, if k = i ×m, i = 0, 1, 2, …
0, otherwise

AR(1)m: ACF and PACF

drawing and guess what is AR(2)m

Seasonal MA(1)m model

Seasonal MA(1)m model takes the following form

Yt = c + Θ1εt−m + εt

with m the seasonal frequency, e.g., m = 12 for monthly

In the form of B operator

Yt = c + (1 + Θ1B

Compared to the non-seasonal MA(1)

Yt = c + θ1εt−1 + εt

Note the use of the notations Θ1 and θ1
ACF of MA(1)m

, if k = m

0, k ≥ 1 and k 6= m

MA(1)m: ACF and PACF

drawing and guess what is MA(2)m

Seasonal ARMA(P ,Q)m model

The general form of ARMA(P,Q)m model(

where the error terms εts are white noise with mean zero and
variance σ2.

Conditions for stationarity and invertibility are the same as for
ARMA. E.g., AR(1)m is stationary if |Φ1| < 1 For AR(P)m (which is ARMA(P, 0)m): ACF dies down at lags im, i = 0, 1, 2, ... and PACF cuts off after lag Pm. For MA(Q)m (which is ARMA(0,Q)m): ACF cuts off after lag Qm and PACF dies down at lags im, i = 0, 1, 2, .... For ARMA(P,Q)m: both ACF and PACF die down at lags im, i = 0, 1, 2, .... ARMA(P ,Q)m: ACF and PACF Mixed Seasonal ARMA(p, q)(P ,Q)m In practice, many time series can be modeled well by a mixed seasonal ARMA model  Yt = c + where the error terms εts are white noise with mean zero and variance σ2. For example, an ARMA(0, 1)(1, 0)12 model is a combination of a seasonal AR(1)12 and a non-seasonal MA(1) Yt − Φ1Yt−12 = c + εt + θ1εt−1 The behavior of ACF and PACF is a combination of behavior of the seasonal and nonseasonal parts of the model. Mixed Seasonal ARMA(p, q)(P ,Q)m: Example We can conclude that the seasonal part is MA(1)12. The orders p and q of the non-seasonal part ARMA(p, q) are not clear - model selection criteria are needed Seasonal ARIMA(p, d , q)(P ,D,Q)m Model Seasonal ARMA and mixed seasonal ARMA require stationarity For non-stationary time series Yt , by taking d-order difference d(Yt) = (1− B)d(Yt) and D-order seasonal difference m(Yt) = (1− B we can arrive at a transformed time series Zt which is stationary By combining difference and seasonal difference with mixed seasonal ARMA, we have a very general seasonal autoregressive integrated moving average (Seasonal ARIMA) model, also called Seasonal Box-Jenkins models. Seasonal Box-Jenkins models ARIMA(p, d , q)(P,D,Q)m models where m = number of seasonal period (e.g. m = 12). (1 − φ1B − φ2B 2 − · · · − φpB 2m − · · · − ΦPB )(1 − B)d (1 − Bm)DYt =c + (1 + θ1B + θ2B + · · · + θqB + · · · + ΘQB Seasonal Box-Jenkins models: Example The above is an ARIMA(1, 1, 1)(1, 1, 1)4 model (with c = 0) Seasonal Box-Jenkins models ARIMA(1, 0, 0)(0, 1, 1)12 for monthly data ARIMA(1, 0, 0)(0, 1, 1)12 for monthly data: (1− φ1B)(1− B12)Yt = c + (1 + Θ1B12)εt This is equivalent to Yt − Yt−12 = c + φ1(Yt−1 − Yt−13) + εt + Θ1εt−12 Seasonal Box-Jenkins models ARIMA(1, 0, 0)(1, 0, 0)12 models (1− φ1B)(1− Φ1B12)Yt = c + εt Or write it out Yt = c + φ1Yt−1 + Φ1Yt−12 − φ1Φ1Yt−13 + εt This is because informally (1− φ1B)(1− Φ1B12) = 1− φ1B − Φ1B12 + φ1Φ1B13 (1− φ1B)(1− Φ1B12)Yt = Yt − φ1Yt−1 − Φ1Yt−12 + φ1Φ1Yt−13 Seasonal Box-Jenkins models ARIMA(1, 0, 0)(1, 0, 0)12 models (1− φ1B)(1− Φ1B12)Yt = c + εt Or write it out Yt = c + φ1Yt−1 + Φ1Yt−12 − φ1Φ1Yt−13 + εt This is because informally (1− φ1B)(1− Φ1B12) = 1− φ1B − Φ1B12 + φ1Φ1B13 (1− φ1B)(1− Φ1B12)Yt = Yt − φ1Yt−1 − Φ1Yt−12 + φ1Φ1Yt−13 This is how Seasonal Box-Jenkins models ARIMA(p, d , q)(P,D,Q)m models ARIMA(1, 1, 1)(1, 1, 0)12 model: (1− φ1B)(1− Φ1B12)(1− B)(1− B12)Yt = c + (1 + θ1B)εt First denote Zt = (1−B)(1−B12)Yt . For this new time series Zt , the model is (1− φ1B)(1− Φ1B12)Zt = c + (1 + θ1B)εt (1− φ1B − Φ1B12 + φ1Φ1B13)Zt = c + (1 + θ1B)εt Zt = φ1BZt + Φ1B 12Zt − φ1Φ1B13Zt + c + εt + θ1Bεt Zt = φ1Zt−1 + Φ1Zt−12 − φ1Φ1Zt−13 + c + εt + θ1εt−1 Seasonal Box-Jenkins models ARIMA(p, d , q)(P,D,Q)m models ARIMA(1, 1, 1)(1, 1, 0)12 model: (1− φ1B)(1− Φ1B12)(1− B)(1− B12)Yt = c + (1 + θ1B)εt First denote Zt = (1−B)(1−B12)Yt . For this new time series Zt , the model is (1− φ1B)(1− Φ1B12)Zt = c + (1 + θ1B)εt (1− φ1B − Φ1B12 + φ1Φ1B13)Zt = c + (1 + θ1B)εt Zt = φ1BZt + Φ1B 12Zt − φ1Φ1B13Zt + c + εt + θ1Bεt Zt = φ1Zt−1 + Φ1Zt−12 − φ1Φ1Zt−13 + c + εt + θ1εt−1 Seasonal Box-Jenkins models ARIMA(p, d , q)(P,D,Q)m models ARIMA(1, 1, 1)(1, 1, 0)12 model: (1− φ1B)(1− Φ1B12)(1− B)(1− B12)Yt = c + (1 + θ1B)εt First denote Zt = (1−B)(1−B12)Yt . For this new time series Zt , the model is (1− φ1B)(1− Φ1B12)Zt = c + (1 + θ1B)εt (1− φ1B − Φ1B12 + φ1Φ1B13)Zt = c + (1 + θ1B)εt Zt = φ1BZt + Φ1B 12Zt − φ1Φ1B13Zt + c + εt + θ1Bεt Zt = φ1Zt−1 + Φ1Zt−12 − φ1Φ1Zt−13 + c + εt + θ1εt−1 Seasonal Box-Jenkins models ARIMA(p, d , q)(P,D,Q)m models ARIMA(1, 1, 1)(1, 1, 0)12 model: (1− φ1B)(1− Φ1B12)(1− B)(1− B12)Yt = c + (1 + θ1B)εt First denote Zt = (1−B)(1−B12)Yt . For this new time series Zt , the model is (1− φ1B)(1− Φ1B12)Zt = c + (1 + θ1B)εt (1− φ1B − Φ1B12 + φ1Φ1B13)Zt = c + (1 + θ1B)εt Zt = φ1BZt + Φ1B 12Zt − φ1Φ1B13Zt + c + εt + θ1Bεt Zt = φ1Zt−1 + Φ1Zt−12 − φ1Φ1Zt−13 + c + εt + θ1εt−1 Seasonal Box-Jenkins models ARIMA(p, d , q)(P,D,Q)m models ARIMA(1, 1, 1)(1, 1, 0)12 model: (1− φ1B)(1− Φ1B12)(1− B)(1− B12)Yt = c + (1 + θ1B)εt Finally from Zt = φ1Zt−1 + Φ1Zt−12 − φ1Φ1Zt−13 + c + εt + θ1εt−1 Zt =(1− B)(1− B12)Yt = Yt − Yt−1 − Yt−12 + Yt−13 =(Yt − Yt−1)− (Yt−12 − Yt−13) (Yt − Yt−1)− (Yt−12 − Yt−13) =c + φ1 [(Yt−1 − Yt−2)− (Yt−13 − Yt−14)] + Φ1 [(Yt−12 − Yt−13)− (Yt−24 − Yt−25)] − φ1Φ1 [(Yt−13 − Yt−14)− (Yt−25 − Yt−26)] + εt + θ1εt−1 Seasonal Box-Jenkins models ARIMA(p, d , q)(P,D,Q)m models ARIMA(1, 1, 1)(1, 1, 0)12 model: (1− φ1B)(1− Φ1B12)(1− B)(1− B12)Yt = c + (1 + θ1B)εt Finally from Zt = φ1Zt−1 + Φ1Zt−12 − φ1Φ1Zt−13 + c + εt + θ1εt−1 Zt =(1− B)(1− B12)Yt = Yt − Yt−1 − Yt−12 + Yt−13 =(Yt − Yt−1)− (Yt−12 − Yt−13) (Yt − Yt−1)− (Yt−12 − Yt−13) =c + φ1 [(Yt−1 − Yt−2)− (Yt−13 − Yt−14)] + Φ1 [(Yt−12 − Yt−13)− (Yt−24 − Yt−25)] − φ1Φ1 [(Yt−13 − Yt−14)− (Yt−25 − Yt−26)] + εt + θ1εt−1 Seasonal Box-Jenkins models ARIMA(p, d , q)(P,D,Q)m models ARIMA(1, 1, 1)(1, 1, 0)12 model: (1− φ1B)(1− Φ1B12)(1− B)(1− B12)Yt = c + (1 + θ1B)εt Finally from Zt = φ1Zt−1 + Φ1Zt−12 − φ1Φ1Zt−13 + c + εt + θ1εt−1 Zt =(1− B)(1− B12)Yt = Yt − Yt−1 − Yt−12 + Yt−13 =(Yt − Yt−1) − (Yt−12 − Yt−13) (Yt − Yt−1) − (Yt−12 − Yt−13) =c + φ1 [(Yt−1 − Yt−2) − (Yt−13 − Yt−14)] + Φ1 [(Yt−12 − Yt−13)− (Yt−24 − Yt−25)] − φ1Φ1 [(Yt−13 − Yt−14)− (Yt−25 − Yt−26)] + εt + θ1εt−1 Seasonal ARIMA models First and seasonal differencing In our example for the log visitors series, we saw that seasonally differencing is not enough to make the series stationary. We can then consider the transform: (1− B12)(1− B)Yt = (Yt − Yt−1)− (Yt−12 − Yt−13) (Yt − Yt−1)− (Yt−12 − Yt−13) = (Yt − Yt−12)− (Yt−1 − Yt−13) = (1− B12)Yt − (1− B12)Yt−1 denote Zt = (1− B12)Yt : = Zt − Zt−1 = (1− B)Zt = (1− B)(1− B12)Yt (1− B12)(1− B)Yt = (1− B)(1− B12)Yt Seasonal ARIMA models First and seasonal differencing In our example for the log visitors series, we saw that seasonally differencing is not enough to make the series stationary. We can then consider the transform: (1− B12)(1− B)Yt = (Yt − Yt−1)− (Yt−12 − Yt−13) (Yt − Yt−1)− (Yt−12 − Yt−13) = (Yt − Yt−12)− (Yt−1 − Yt−13) = (1− B12)Yt − (1− B12)Yt−1 denote Zt = (1− B12)Yt : = Zt − Zt−1 = (1− B)Zt = (1− B)(1− B12)Yt (1− B12)(1− B)Yt = (1− B)(1− B12)Yt Seasonal ARIMA models First and seasonal differencing In our example for the log visitors series, we saw that seasonally differencing is not enough to make the series stationary. We can then consider the transform: (1− B12)(1− B)Yt = (Yt − Yt−1)− (Yt−12 − Yt−13) (Yt − Yt−1)− (Yt−12 − Yt−13) = (Yt − Yt−12)− (Yt−1 − Yt−13) = (1− B12)Yt − (1− B12)Yt−1 denote Zt = (1− B12)Yt : = Zt − Zt−1 = (1− B)Zt = (1− B)(1− B12)Yt (1− B12)(1− B)Yt = (1− B)(1− B12)Yt Seasonal ARIMA models First and seasonal differencing In our example for the log visitors series, we saw that seasonally differencing is not enough to make the series stationary. We can then consider the transform: (1− B12)(1− B)Yt = (Yt − Yt−1)− (Yt−12 − Yt−13) (Yt − Yt−1)− (Yt−12 − Yt−13) = (Yt − Yt−12)− (Yt−1 − Yt−13) = (1− B12)Yt − (1− B12)Yt−1 denote Zt = (1− B12)Yt : = Zt − Zt−1 = (1− B)Zt = (1− B)(1− B12)Yt (1− B12)(1− B)Yt = (1− B)(1− B12)Yt Seasonal ARIMA models First and seasonal differencing In our example for the log visitors series, we saw that seasonally differencing is not enough to make the series stationary. We can then consider the transform: (1− B12)(1− B)Yt = (Yt − Yt−1)− (Yt−12 − Yt−13) (Yt − Yt−1)− (Yt−12 − Yt−13) = (Yt − Yt−12)− (Yt−1 − Yt−13) = (1− B12)Yt − (1− B12)Yt−1 denote Zt = (1− B12)Yt : = Zt − Zt−1 = (1− B)Zt = (1− B)(1− B12)Yt (1− B12)(1− B)Yt = (1− B)(1− B12)Yt Seasonal ARIMA models First and seasonal differencing In our example for the log visitors series, we saw that seasonally differencing is not enough to make the series stationary. We can then consider the transform: (1− B12)(1− B)Yt = (Yt − Yt−1)− (Yt−12 − Yt−13) (Yt − Yt−1)− (Yt−12 − Yt−13) = (Yt − Yt−12)− (Yt−1 − Yt−13) = (1− B12)Yt − (1− B12)Yt−1 denote Zt = (1− B12)Yt : = Zt − Zt−1 = (1− B)Zt = (1− B)(1− B12)Yt (1− B12)(1− B)Yt = (1− B)(1− B12)Yt Seasonal Box-Jenkins models ARIMA(p, d , q)(P,D,Q)m models ARIMA(0, 0, 0)(P, 0, 0) Sample autocorrelations die down for lags m, 2m, 3m, etc. Sample partial autocorrelations cut off at lag Pm. ARIMA(0, 0, 0)(0, 0,Q) Sample autocorrelations cuts off at lag Qm. Sample partial autocorrelations die down for lags m, 2m, 3m, ARIMA(0, 0, 0)(0, 1, 0) Sample autocorrelations and partial autocorrelations die down very slowly for lags m, 2m, 3m, etc. Seasonal ARIMA models Lecture09 Example01.py Carefully read the scripts in Lecture09 Example01.py The concluded model is (1−φ1B−φ2B2−φ3B3−φ4B4−φ5B5)(1−B12)Zt = εt +Θ1B12εt (1− φ1B − φ3B3 − φ5B5)(1− B12)Zt = εt + Θ1B12εt where Zt is the quartic root data, i.e., Zt = Y Seasonal ARIMA models Lecture09 Example01.py Carefully read the scripts in Lecture09 Example01.py The concluded model is (1−φ1B −φ2B2 −φ3B3 −φ4B4 −φ5B5)(1−B12)Zt = εt +Θ1B12εt (1 − φ1B − φ3B3 − φ5B5)(1 − B12)Zt = εt + Θ1B12εt where Zt is the quartic root data, i.e., Zt = Yt Seasonal ARIMA models First and seasonal differencing Seasonal ARIMA models First and seasonal differencing Seasonal ARIMA models First and seasonally differenced log visitors: estimation ARIMA(2, 1, 2)(0, 1, 1)12 model: (1−φ1B−φ2B2)(1−B)(1−B12)Yt = c+(1+θ1B+θ2B2)(1+Θ1B12)εt Estimated coefficients (using R): ar1 ar2 ma1 ma2 sma1 -0.7817 -0.3154 -0.0300 -0.4007 -0.7471 s.e. 0.2212 0.1227 0.2213 0.1909 0.1073 log likelihood=178.99, AIC=-345.97, AICc=-345.08, BIC=-330.28. Seasonal ARIMA models ARIMA(2, 1, 2)(0, 1, 1)12 model: residuals Seasonal ARIMA models ARIMA(2, 1, 2)(0, 1, 1)12 model: forecasts Seasonal ARIMA models Intercept terms induce permanent trends. Only seasonally difference once. Usually only either one seasonal AR or MA term is needed. Seasonal AR terms are often used when the lag m sample autocorrelation terms are positive. Seasonal MA terms are often used when the lag m sample autocorrelation terms are negative. Forecasting combinations Introduction Classical reference: Bates, J. M., and C. W. J. Granger (1969). The combination of forecasts, Operational Research Quarterly, 20, 451–468. They provide the following illustration: Forecasting combinations Introduction It is possible to combine unbiased forecasts ŷ T+1|T from models i = 1, . . . ,m. The models can be various ARIMA type of models or a set of ARIMA models, HW exponential smoothing models and regression models for example. T+1|T , i = 1, . . . ,m could also be m expert forecasts. Forecasting combinations The forecasts can be combined as follows ŷ cT+1|T = The simplest way is to set wi = , then you are using a simple average. Simple averages often work surprisingly well. We often use convex combinations, that is 0 ≤ w ≤ 1. The question is how to combine forecasts “optimally”? Forecasting combinations Variance reduction: example of two forecasts (optional) If you have two unbiased forecasts ŷ T+1|T and ŷ T+1|T with the corresponding variances σ21 and σ 2, then we can combine them ŷ cT+1|T = wŷ T+1|T + (1− w)ŷ The variance of the combined forecast will be 2σ21 + (1− w) 2σ22 + 2ρwσ1(1− w)σ2. It will have minimum at σ22 − ρσ1σ2 2 − 2ρσ1σ2 where w is the “optimal” value. In case where ŷ T+1|T and ŷ are uncorrelated (ρ = 0), then w = σ22/(σ 2), which is no greater than the smaller of the two individual variances. Forecasting combinations Empirical weights (optional) Unfortunately we dont know the actual σ21 and σ design a way to estimate them. One-step ahead forecasts over a sample T + 1 observations. Dividing the T + 1 observations into an initial estimation (regression) subsample (e.g., from time 1 to t0) and a second evaluation (prediction) subsample (from t0 + 1 to T + 1). The first subsample enables you to estimate the parameters of each model. In the second subsample, the forecasting performance of each model can be evaluated. Each model’s performance will differ from period to period: e1,t0+1, e1,t0+2, ..., e1,T e2,t0+1, e2,t0+2, ..., e2,T Forecasting combinations Empirical weights (optional) We are going to use the estimates (many possibilities) e21,t ; for t1 = t0 + 1, ...,T e22,t ; for t1 = t0 + 1, ...,T In general, we can define The simplest version of such adaptive weights is, for t1 = t0 + 1, ...,T , Forecasting combinations Empirical weights (optional) At the time T , we can have the simplest version of such adaptive weights Or updating as (a clever way) w∗T+1 = αw T + (1− α) = αw∗T + (1− α)wT+1 Forecasting combinations There is extensive empirical evidence in favour of combinations as a forecasting strategy. Forecasting combinations offer diversification gains that make them very useful compared to relying on a single model, as we have just seen. There may be structural breaks in the data, making it plausible that combining models with different levels adaptability will lead to better results than relying on a single Forecasting combinations Even without structural breaks, individuals models may be subject to misspecification bias: it is implausible that a single model dominates all others at all time periods. An additional argument for combining forecasts is that predictions from different forecasters may have been constructed under different loss functions. Forecasting combinations Estimation errors are a serious problem for obtaining the combination weights. Simple averages are often better. Similarly, nonstationarities can cause instabilities on the 程序代写 CS代考 加微信: powcoder QQ: 1823890830 Email: powcoder@163.com