Basics AR process MA process ARMA process Stationarity ARIMA process
CORPFIN 2503 – Business Data Analytics:
Time-series analysis
£ius
Week 9: October 5th, 2021
£ius CORPFIN 2503, Week 9 1/65
Basics AR process MA process ARMA process Stationarity ARIMA process
Outline
Basics
AR process
MA process
ARMA process
Stationarity
ARIMA process
£ius CORPFIN 2503, Week 9 2/65
Basics AR process MA process ARMA process Stationarity ARIMA process
Introduction
Time-series process is a series of data points recorded in time
order where the time interval between any adjacent observations is
the same.
For example, IBM monthly stock prices (�xed interval is a month).
One should have one observation per �xed interval of time for time
series process.
£ius CORPFIN 2503, Week 9 3/65
Basics AR process MA process ARMA process Stationarity ARIMA process
Example: IBM
£ius CORPFIN 2503, Week 9 4/65
Basics AR process MA process ARMA process Stationarity ARIMA process
Example: IBM II
£ius CORPFIN 2503, Week 9 5/65
Basics AR process MA process ARMA process Stationarity ARIMA process
Example: IBM III
£ius CORPFIN 2503, Week 9 6/65
Basics AR process MA process ARMA process Stationarity ARIMA process
Example: IBM IV
£ius CORPFIN 2503, Week 9 7/65
Basics AR process MA process ARMA process Stationarity ARIMA process
Example: IBM V
£ius CORPFIN 2503, Week 9 8/65
Basics AR process MA process ARMA process Stationarity ARIMA process
Introduction II
Suppose we are given the data for the 2003-2020 period.
Forecasting would help us to determine which scenario is the most
plausible for the 2021-2023 period.
I.e., forecasting would tell us how to extend the line beyond 2020
year given the data for the 2003-2020 period and no other
information.
£ius CORPFIN 2503, Week 9 9/65
Basics AR process MA process ARMA process Stationarity ARIMA process
Introduction III
The aim of time-series analysis is to model the patterns to use for
forecasting.
Phases of time-series analysis:
1. Descriptive analysis
2. Modeling
3. Forecasting the future values.
£ius CORPFIN 2503, Week 9 10/65
Basics AR process MA process ARMA process Stationarity ARIMA process
Introduction IV
In the descriptive phase, we should understand the nature of the
time series:
• about what the trend looks like, upward or downward, and
whether
• there is any seasonal trend in the process.
In the modeling phase, one should model the inherent properties of
the time-series data.
In the forecasting phase, one predicts the future values of a variable
of interest using its historical values:
Yt = f(Yt−1, Yt−2, Yt−3, . . . , Y0).
In this course, we will use ARIMA (auto-regressive integrated
moving average) methods to forecast.
£ius CORPFIN 2503, Week 9 11/65
Basics AR process MA process ARMA process Stationarity ARIMA process
AR process
AR (auto-regressive) process is where the current values of the
series depend on their previous values.
It is regression on itself (lagged values).
The AR process is denoted by AR(p), where p is the order of the
auto-regressive process.
p determines on how many previous values the current value of the
series depends.
AR process re�ects long-term trends.
£ius CORPFIN 2503, Week 9 12/65
Basics AR process MA process ARMA process Stationarity ARIMA process
AR process II
The general format of AR(1) process is:
Yt − µ = β(Yt−1 − µ) + ϵt or
Yt = µ+ β(Yt−1 − µ) + ϵt.
where µ is mean.
If µ = 0 then AR(1) process is:
Yt = βYt−1 + ϵt.
For AR(1) model, SAS procedure ARIMA provides estimates for
MU and AR(1):
Yt = µ+ AR(1)× (Yt−1 − µ) + ϵt.
£ius CORPFIN 2503, Week 9 13/65
Basics AR process MA process ARMA process Stationarity ARIMA process
AR process III
Let’s regress IBM sales growth on its lagged values. First, we need
to generate sales growth variable:
data work.ibm2;
set work.ibm2;
lag_sale=lag(sale);
sale_growth=sale/lag_sale-1;
lag_sale_growth=lag(sale_growth);
run;
£ius CORPFIN 2503, Week 9 14/65
Basics AR process MA process ARMA process Stationarity ARIMA process
AR process IV
Let’s look at the �rst few observations and time series plot of the
sales growth:
proc print data=work.ibm2 label;
var fyear tic sale sale_growth lag_sale_growth;
run;
symbol1 color=red interpol=join;
proc gplot data=work.ibm2;
plot sale_growth*fyear / hminor=0;
run;
quit;
£ius CORPFIN 2503, Week 9 15/65
Basics AR process MA process ARMA process Stationarity ARIMA process
AR process V
£ius CORPFIN 2503, Week 9 16/65
Basics AR process MA process ARMA process Stationarity ARIMA process
AR process VI
£ius CORPFIN 2503, Week 9 17/65
Basics AR process MA process ARMA process Stationarity ARIMA process
AR process VII
Now let’s estimate AR(1) model for IBM sales growth, using SAS
procedure ARIMA::
proc arima data=work.ibm2;
identify var=sale_growth;
estimate p=1 q=0;
run;
quit;
£ius CORPFIN 2503, Week 9 18/65
Basics AR process MA process ARMA process Stationarity ARIMA process
AR process VIII: SAS output
£ius CORPFIN 2503, Week 9 19/65
Basics AR process MA process ARMA process Stationarity ARIMA process
AR process IX: SAS output
£ius CORPFIN 2503, Week 9 20/65
Basics AR process MA process ARMA process Stationarity ARIMA process
AR process X: SAS output
£ius CORPFIN 2503, Week 9 21/65
Basics AR process MA process ARMA process Stationarity ARIMA process
AR process XI: SAS output
£ius CORPFIN 2503, Week 9 22/65
Basics AR process MA process ARMA process Stationarity ARIMA process
AR process XII: SAS output
£ius CORPFIN 2503, Week 9 23/65
Basics AR process MA process ARMA process Stationarity ARIMA process
AR process XIII: SAS output
£ius CORPFIN 2503, Week 9 24/65
Basics AR process MA process ARMA process Stationarity ARIMA process
AR process XIV
The resulting AR(1) model is:
Yt = µ+ β(Yt−1 − µ) + ϵt,
Yt = 0.09664 + 0.46573(Yt−1 − 0.09664) + ϵt.
You might wonder how these results are di�erent from OLS. Let’s
have a look:
proc reg data=work.ibm2;
model sale_growth=lag_sale_growth;
run;
£ius CORPFIN 2503, Week 9 25/65
Basics AR process MA process ARMA process Stationarity ARIMA process
AR process XV: SAS output
The resulting model:
Yt = 0.04764 + 0.46574Yt−1 + ϵt.
£ius CORPFIN 2503, Week 9 26/65
Basics AR process MA process ARMA process Stationarity ARIMA process
AR process XVI
Let’s compare the models:
AR(1): Yt = 0.09664 + 0.46573(Yt−1 − 0.09664) + ϵt.
OLS: Yt = 0.04764 + 0.46574Yt−1 + ϵt.
Intercepts are di�erent but slopes are very similar.
We should not use OLS regressions to estimate AR models.
£ius CORPFIN 2503, Week 9 27/65
Basics AR process MA process ARMA process Stationarity ARIMA process
AR process XVII
AR(2) process is:
Yt = µ+ β1(Yt−1 − µ) + β2(Yt−2 − µ) + ϵt
Let’s estimate AR(2) model for IBM sales growth, using SAS
procedure ARIMA:
proc arima data=work.ibm2;
identify var=sale_growth;
estimate p=2 q=0;
run;
quit;
£ius CORPFIN 2503, Week 9 28/65
Basics AR process MA process ARMA process Stationarity ARIMA process
AR process XVIII: SAS output
The resulting model:
Yt = 0.10187+0.34549(Yt−1−0.10187)+0.27025(Yt−2−0.10187)+ϵt.
£ius CORPFIN 2503, Week 9 29/65
Basics AR process MA process ARMA process Stationarity ARIMA process
AR process XIX
So which model is the best for IBM sales growth: AR(1), AR(2), or
perhaps AR(3)?
It is not easy to determine the order of an AR process by simply
looking at the shape of the time-series curve alone.
Statistical tests are used to determine the order of AR process.
£ius CORPFIN 2503, Week 9 30/65
Basics AR process MA process ARMA process Stationarity ARIMA process
MA process
A moving average (MA) process is a time-series process where
the current value and the previous values in the series are almost
the same.
But the current deviation in the series depends upon the previous
error or residual or shock (ϵt, ϵt−1. . . ).
ϵt is considered to be an unobserved term.
E.g., if we consider monthly stock returns, the ϵt includes shocks a
stock price is subject to:
• unexpected changes to macroeconomic environment
• investor sentiment
• �rm operating performance
• etc.
£ius CORPFIN 2503, Week 9 31/65
Basics AR process MA process ARMA process Stationarity ARIMA process
MA process II
In an AR process, Yt depends on the previous values of Y
(e.g.,Yt−1), whereas in the MA process, the previous values of Y
are irrelevant.
MA process is about the short-term shocks in the series.
MA is not related to the long-term trends, like in an AR process.
In the MA process, the current value of the series is a factor of the
previous errors.
The MA process is denoted by MA(q), where q determines how
many previous error values have an e�ect on the current value of an
MA series.
£ius CORPFIN 2503, Week 9 32/65
Basics AR process MA process ARMA process Stationarity ARIMA process
MA process: notation
The �rst order MA (MA(1)) process:
Yt = µ+ ϵt −MA(1)ϵt−1, where
µ is mean
MA(1) is the factor or the quanti�ed impact of ϵt−1 on the
current deviation
ϵt is the residual at time t.
£ius CORPFIN 2503, Week 9 33/65
Basics AR process MA process ARMA process Stationarity ARIMA process
MA process III
Let’s estimate the MA(1) model for monthly returns on IBM stock:
proc arima data=work.ibm;
identify var=ret;
estimate p=0 q=1;
run;
quit;
£ius CORPFIN 2503, Week 9 34/65
Basics AR process MA process ARMA process Stationarity ARIMA process
MA process IV
SAS output:
The resulting model:
Yt = 0.006 + ϵt + 0.028ϵt−1.
£ius CORPFIN 2503, Week 9 35/65
Basics AR process MA process ARMA process Stationarity ARIMA process
MA process V
Let’s estimate the MA(2) model for monthly returns on IBM stock:
proc arima data=work.ibm;
identify var=ret;
estimate p=0 q=2;
run;
quit;
£ius CORPFIN 2503, Week 9 36/65
Basics AR process MA process ARMA process Stationarity ARIMA process
MA process VI
SAS output:
The resulting model:
Yt = 0.006 + ϵt + 0.033ϵt−1 + 0.053ϵt−2.
£ius CORPFIN 2503, Week 9 37/65
Basics AR process MA process ARMA process Stationarity ARIMA process
MA process VII
The shape of any AR(p) and MA(q) series curves depend upon the
actual parameters.
Even if the order of two MA(q) series is the same, their plot shapes
can di�er based upon the actual parameter values in their
respective equations.
It is not easy to determine the order of an MA process by simply
looking at the shape of the time-series curve alone.
Statistical tests are used to determine the order of MA process.
£ius CORPFIN 2503, Week 9 38/65
Basics AR process MA process ARMA process Stationarity ARIMA process
ARMA process
An ARMA process is a process that shows the properties of an
auto-regressive process and a moving average process.
In an ARMA time-series process, the current value of the series
depends on its previous values.
The small deviations from the mean value in an ARMA process are
a factor of the previous errors.
An ARMA process is a series with both:
• long-term trend (AR process) and
• short-term shocks (MA process).
£ius CORPFIN 2503, Week 9 39/65
Basics AR process MA process ARMA process Stationarity ARIMA process
ARMA process: notation
ARMA(p,q) is the general notation for an ARMA process, where:
p is the order of the AR process and
q is the order of the MA process.
E.g., ARMA(1,1):
Yt = µ+ a1(Yt−1 − µ) + ϵt − b1ϵt−1,where
a1 is the quanti�ed impact of (Yt−1 − µ) on Yt
b1 is the quanti�ed impact of ϵt−1 on Yt.
Similarly, ARMA(2,2):
Yt = µ+ a1(Yt−1 − µ) + a2(Yt−2 − µ) + ϵt − b1ϵt−1 − b2ϵt−2.
£ius CORPFIN 2503, Week 9 40/65
Basics AR process MA process ARMA process Stationarity ARIMA process
ARMA process II
Let’s estimate ARMA(1,1) model for IBM sales growth:
proc arima data=work.ibm2;
identify var=sale_growth;
estimate p=1 q=1;
run;
quit;
£ius CORPFIN 2503, Week 9 41/65
Basics AR process MA process ARMA process Stationarity ARIMA process
ARMA process III
The resulting model:
Yt = 0.16419 + 1(Yt−1 − 0.16419) + ϵt − 0.84763ϵt−1.
£ius CORPFIN 2503, Week 9 42/65
Basics AR process MA process ARMA process Stationarity ARIMA process
ARMA process IV
Let’s estimate ARMA(2,2) model for IBM sales growth:
proc arima data=work.ibm2;
identify var=sale_growth;
estimate p=2 q=2;
run;
quit;
£ius CORPFIN 2503, Week 9 43/65
Basics AR process MA process ARMA process Stationarity ARIMA process
ARMA process V
The resulting model:
Yt = 0.183+1.999(Yt−1−0.183)−1(Yt−2−0.183)+ϵt−1.871ϵt−1+0.878ϵt−2.
£ius CORPFIN 2503, Week 9 44/65
Basics AR process MA process ARMA process Stationarity ARIMA process
ARMA process VI
At this stage, we cannot tell which model�ARMA(1,1) or
ARMA(2,2)��ts the data better.
This will be determined by statistical tests.
It is also possible that AR(p) model or MA(q) model are superior to
ARMA(p,q) model.
£ius CORPFIN 2503, Week 9 45/65
Basics AR process MA process ARMA process Stationarity ARIMA process
Stationarity
If we want to use ARIMA models for forecasting, our time series
needs to be stationary.
Stationary time series has no trend and no systematic change in
variance.
In other words, in the stationary time-series process, the mean and
variance hover around a single value.
£ius CORPFIN 2503, Week 9 46/65
Basics AR process MA process ARMA process Stationarity ARIMA process
Stationarity II
Source: Konasani, V. R. and Kadre, S. (2015). �Practical Business
Analytics Using SAS: A Hands-on Guide�, p. 454.
£ius CORPFIN 2503, Week 9 47/65
Basics AR process MA process ARMA process Stationarity ARIMA process
Stationarity III
Source: Konasani, V. R. and Kadre, S. (2015). �Practical Business
Analytics Using SAS: A Hands-on Guide�, p. 455.
£ius CORPFIN 2503, Week 9 48/65
Basics AR process MA process ARMA process Stationarity ARIMA process
Stationarity IV
If time series’ mean or variance is increasing over time (exploding),
the time series is likely to be non-stationary.
One needs to transform it in order to achieve stationarity.
The easiest transformation is the �rst di�erence of the time series:
∆Yt = Yt − Yt−1.
Then use ∆Yt instead of Yt in the analysis.
Some series may not be stationary even after the �rst
di�erentiation.
Then one should try to use the second di�erentiation
(∆Yt −∆Yt−1).
£ius CORPFIN 2503, Week 9 49/65
Basics AR process MA process ARMA process Stationarity ARIMA process
Stationarity V
Let’s test whether IBM sales data is stationary time series using
augmented Dickey-Fuller (DF) unit root test:
proc arima data=work.ibm2;
identify var= sale stationarity=(DICKEY);
run;
quit;
£ius CORPFIN 2503, Week 9 50/65
Basics AR process MA process ARMA process Stationarity ARIMA process
Stationarity VI
SAS output:
£ius CORPFIN 2503, Week 9 51/65
Basics AR process MA process ARMA process Stationarity ARIMA process
Stationarity VII
Figure as part of SAS output:
`Spikes’ outside 2 standard error bands are statistically signi�cant.
£ius CORPFIN 2503, Week 9 52/65
Basics AR process MA process ARMA process Stationarity ARIMA process
Stationarity VIII
From the table, we focus on the Single Mean portion.
The DF test has two sets of P-values:
1. The �rst set of P-values (Pr