ARMA Models
ARMA Models
MAS 640
1/24/2018
Chicken Price
Time
ch
ic
ke
n
2010 2011 2012 2013 2014 2015 2016 2017
8
0
9
0
1
0
0
1
1
0
1
2
0
1
3
0
CREF Data Example
Time
C
R
E
F
0 100 200 300 400 500
1
7
0
1
8
0
1
9
0
2
0
0
2
1
0
2
2
0
MA(q) Model
I MA(q) models have 0 autocorrelation beyond lag q
ACF Plots for MA(q) Processes
0 5 10 15 20 25 30
0
.0
0
.2
0
.4
0
.6
Lag
A
C
F
0 5 10 15 20 25 30
−
0
.1
0
.1
0
.3
0
.5
Lag
A
C
F
0 5 10 15 20 25 30
−
0
.5
−
0
.3
−
0
.1
Lag
A
C
F
0 5 10 15 20 25 30
0
.0
0
.2
0
.4
0
.6
Lag
A
C
F
AR(p) Model
I Autocorrelation for AR(p) process decays over time, but does
not cut off to 0
ACF Plots for AR(p) Processes
0 5 10 15 20 25 30
0
.0
0
.2
0
.4
0
.6
0
.8
Lag
A
C
F
0 5 10 15 20 25 30
0
.0
0
.2
0
.4
0
.6
0
.8
Lag
A
C
F
0 5 10 15 20 25 30
−
0
.5
0
.0
0
.5
Lag
A
C
F
0 5 10 15 20 25 30
0
.0
0
.2
0
.4
0
.6
Lag
A
C
F
Which Model Generated These Plots?
0 5 10 15 20 25 30 35
−
0
.4
−
0
.2
0
.0
Lag
A
C
F
0 5 10 15 20 25 30 35
0
.0
0
.2
0
.4
0
.6
Lag
A
C
F
0 5 10 15 20 25 30 35
−
0
.0
4
−
0
.0
2
0
.0
0
0
.0
2
Lag
A
C
F
0 5 10 15 20 25 30 35
0
.0
0
.1
0
.2
0
.3
0
.4
0
.5
Lag
A
C
F
Reading the ACF Plot
AR(p) MA(q)
ACF Tails off Cuts off after lag q
I Knowing this, we can quickly determine. . .
I If we need MA components in our model (and how many)
I If we need AR components
I Unfortunately, due to the nature of ACF for AR processes, we
cannot determine the order, p
Motivation
I The ACF provides a considerable amount of info about the
order of the dependence or MA processes
I Unfortunately, it doesn’t for AR processes
I Need a new function that behaves like the ACF for MA models,
but for use with AR models instead.
I This function is called the partial autocorrelation function
(PACF).
Partial Autocorrelation Function (PACF)
I Basic idea: What is the correlation between Yt and Yt−k
after taking into account Yt−1,Yt−2, · · ·Yt−k+1
I Technically:
1. Regress Yt on Yt−1,Yt−2, · · ·Yt−k+1 and find Ŷt
2. Regress Yt−k on Yt−1,Yt−2, · · ·Yt−k+1 and find Ŷt−k
3. Use correlation between Ŷt and Ŷt−k
I “Partial” because I’m removing everything in the middle. . .
I pacf() for PACF, or acf2() for both ACF and PACF
PACF for AR(p) Models
I Can you guess the AR order, p?
0 5 10 15 20 25 30
0
.0
0
.2
0
.4
0
.6
0
.8
Lag
P
a
rt
ia
l A
C
F
0 5 10 15 20 25 30
0
.0
0
.2
0
.4
0
.6
0
.8
Lag
P
a
rt
ia
l A
C
F
0 5 10 15 20 25 30
−
0
.8
−
0
.4
0
.0
Lag
P
a
rt
ia
l A
C
F
0 5 10 15 20 25 30
0
.0
0
.2
0
.4
0
.6
Lag
P
a
rt
ia
l A
C
F
ACF Plots for MA(q) Processes
I Can you guess the MA order q?
0 10 20 30 40
−
0
.2
0
.0
0
.2
0
.4
0
.6
Lag
P
a
rt
ia
l A
C
F
0 10 20 30 40
−
0
.3
0
−
0
.2
0
−
0
.1
0
0
.0
0
Lag
P
a
rt
ia
l A
C
F
0 10 20 30 40
−
0
.2
0
.0
0
.2
0
.4
Lag
P
a
rt
ia
l A
C
F
0 10 20 30 40
−
0
.2
0
.0
0
.2
0
.4
0
.6
0
.8
Lag
P
a
rt
ia
l A
C
F
Determing AR and MA Components
AR(p) MA(q)
ACF Tails off Cuts off after lag q
PACF Cuts off after lag p Tails off
I Now we are comfortable with these!
I Let’s use them to build more complicated models
Autoregressive Moving Average (ARMA) Process
Suppose {et} is a zero mean white noise process with var(et) = σ2e .
The process
Yt = φ1Yt−1 +φ2Yt−2 + . . .+φpYt−p + et + θ1et−1 + . . .+ θqet−q
is an autoregressive moving average process with AR order p and
MA order q, written as ARMA(p,q).
ARMA(p,q) Process
I Special cases
I ARMA(0, 0):
I ARMA(p,0):
I ARMA(0,q):
Remembering the Goal
Yt = µt + et
I We model µt with standard linear regression techniques
I We are left with the error terms
I These errors are correlated
I We want to model them
I These errors can be correlated with past values of itself (AR) or
past errors (MA)
I We just need to determine which components (AR or MA), and
how many of each, to include in our model
Some ARMA(1, 1) Processes
phi 0.9, theta 0.9
Time
si
m
1
0 20 40 60 80 100
−
5
0
5
phi 0.7, theta 0.5
Time
si
m
2
0 20 40 60 80 100
−
2
0
2
4
phi 0.5, theta −0.5
Time
si
m
3
0 20 40 60 80 100
−
3
−
2
−
1
0
1
2
phi −0.5, theta −0.5
Time
si
m
4
0 20 40 60 80 100
−
4
−
2
0
2
ACF for ARMA(1,1) Processes
0 5 10 15 20 25 30
0
.0
0
.2
0
.4
0
.6
0
.8
phi 0.9, theta 0.9
Lag
A
C
F
0 5 10 15 20 25 30
0
.0
0
.2
0
.4
0
.6
0
.8
phi 0.7, theta 0.5
Lag
A
C
F
0 5 10 15 20 25 30
0
.0
0
.2
0
.4
phi 0.5, theta −0.5
Lag
A
C
F
0 5 10 15 20 25 30
−
0
.6
−
0
.2
0
.2
0
.4
phi −0.5, theta −0.5
Lag
A
C
F
PACF for ARMA(1,1) Processes
0 5 10 15 20 25 30
−
0
.4
0
.0
0
.4
0
.8
Lag
P
a
rt
ia
l A
C
F
phi 0.9, theta 0.9
0 5 10 15 20 25 30
−
0
.2
0
.2
0
.4
0
.6
0
.8
Lag
P
a
rt
ia
l A
C
F
phi 0.7, theta 0.5
0 5 10 15 20 25 30
0
.0
0
.2
0
.4
Lag
P
a
rt
ia
l A
C
F
phi 0.5, theta −0.5
0 5 10 15 20 25 30
−
0
.6
−
0
.4
−
0
.2
0
.0
0
.2
Lag
P
a
rt
ia
l A
C
F
phi −0.5, theta −0.5
Determining Order for ARMA(p, q)
I Unfortunately, no clear “drop-offs” are present in the ACF or
PACF plots
I So we can’t determine p or q if AR and MA components are
both present in model
I We do have the following to guide the process:
AR(p) MA(q) ARMA(p,q)
ACF Tails off Cuts off after lag q Tails off
PACF Cuts off after lag p Tails off Tails off
Fitting an ARMA(p, q) Model
I If the ACF and PACF both tail off, this is evidence of an
ARMA model
I But how to determine p and q?
I “Go fit an ARMA model to this data. . . ”
Fitting ARMA Models in R
I Previously used ar() function to fit an AR model and let it
decide the order p by comparing AIC values
I Need something more comprehensive for ARMA models
I arma()
I arima()
I sarima()
I Each of these accomplish the same thing, some are easier to use
I I will stick with sarima() for the remainder of the course
The sarima() Function
Seasonal Autoregressive Integrated Moving Average
sarima(x, p, d, q, P=0, D=0, Q=0, S=-1, …)
I Where
I x is the data
I p and q are the AR and MA orders (note that you now have to
specify)
I d tells the function to difference the data d times (next class,
use 0 for now)
I The capital letters are seasonal components that we ignore for
now
I x, p, d, and q are required
The ‘sarima.for()’ Function
I sarima() not compatible with predict(), but it does come
with it’s own function for forecasting
sarima.for(x, n.ahead, p, d, q, …)
Examples Using sarima()
Write the code for fitting each of the following models to data x:
1. AR(1) –
2. MA(3) –
3. AR(2) –
4. ARMA(1, 1) –
5. ARMA(2, 1) –
Examples Using sarima.for()
Write the code for forecasting the next 6 time periods for each of
the following models:
1. AR(1) –
2. MA(3) –
3. AR(2) –
4. ARMA(1, 1) –
5. ARMA(2, 1) –