ECON3350/7350 Univariate Time Series – I
Eric Eisenstat
The University of Queensland
Lecture 2
Eric Eisenstat
(School of Economics) ECON3350/7350 Week 2 1 / 18
Univariate Time Series Models
Recommended readings
Author
Title
Chapter
Call No
Enders Verbeek
Applied Econometric Time Series, 4e
A Guide to Modern Econometrics
1, 2
2, 3, 4, 5
HB139 .E55 2015 HB139 .V465 2012
Eric Eisenstat (School of Economics) ECON3350/7350 Week 2 2 / 18
Australian real GDP per capita (seasonally adjusted)
4 x 10
1.8 1.7 1.6 1.5 1.4 1.3 1.2 1.1
1 0.9 0.8
1970 1975 1980
1985 1990
1995 2000 2005 2010 2015 2020
Eric Eisenstat
(School of Economics)
ECON3350/7350
Week 2
3 / 18
A Time Series Process
Definition
A stochastic process is a collection of random variables that are ordered in time.
A time series is a stochastic process
each observation is a random variable
observations evolve in time according to some probabilities
Eric Eisenstat (School of Economics) ECON3350/7350 Week 2 4 / 18
Realisations, Moments
Definition
A realisation is one of a (typically) infinite set of values of
yt, t = 1, . . . , T , randomly generated according to the probability distribution underlying the stochastic process.
We are typically interested in the following moments characterizing the probability distribution:
Mean: μt = E(yt), t = 1,…,T; which can be interpreted as the average value of yt taken over all possible realisations.
Variance: Var(yt) = E((yt − μt)2), t = 1, . . . , T ; i.e., the average of square deviations from the mean.
Covariance: Cov(yt, yt−k) = E((yt − μt)(yt−k − μt−k)), t = k + 1, . . . , T .
Eric Eisenstat (School of Economics) ECON3350/7350 Week 2 5 / 18
Modelling Time-Series Data
What can we do with a set of observations y1, . . . , yT , say quarterly observations of Australian real GDP per capita, from 1973Q3 to 2016Q1?
Eric Eisenstat (School of Economics) ECON3350/7350 Week 2 6 / 18
Modelling Time-Series Data
What can we do with a set of observations y1, . . . , yT , say quarterly observations of Australian real GDP per capita, from 1973Q3 to 2016Q1?
A common goal is to forecast yT+1,…,yT+h.
Eric Eisenstat (School of Economics) ECON3350/7350 Week 2 6 / 18
Modelling Time-Series Data
What can we do with a set of observations y1, . . . , yT , say quarterly observations of Australian real GDP per capita, from 1973Q3 to 2016Q1?
A common goal is to forecast yT+1,…,yT+h.
We shall focus on time series models as forecasting tools (expand later).
Eric Eisenstat (School of Economics) ECON3350/7350 Week 2 6 / 18
Modelling Time Series Data
One simple way to model the time series is with the “regression”:
yt = a0 + a1yt−1 + ut.
This is called the first order auto-regressive model, or AR(1).
We might assume that ut is an error term satisfying standard CR assumptions.
But what CR assumption will certainly not be satisfied?
Eric Eisenstat (School of Economics) ECON3350/7350 Week 2 7 / 18
Estimating the AR(1) Model
What is the implication of having a “stochastic” regressor?
Suppose, for the moment, we knew a0, a1 and wanted to forecast yT+h. A reasonable forecast yT+h could be:
E(yT+h|y1,…,yT)=E(a0 +a1yT+h−1 +uT+h| · ) =a0+a1E(yT+h−1| · )+E(uT+h| · )
=a0+a1(a0+a1E(yT+h−2| · )+E(uT+h−1| · )) .
=a 1+a +a2+···+ah−1+ahy 01111T
Eric Eisenstat
(School of Economics) ECON3350/7350 Week 2 8 / 18
Estimating the AR(1) Model
In the limiting case as h −→ ∞, we get: yT+h−→ a0 if|a1|<1;
1−a1
yT+h −→ indeterminate form (i.e. does not exist) if |a1| ≥ 1.
The AR(1) model with |a1| ≥ 1 is called non-stationary.
can still proceed with OLS estimation, but non-stationarity causes
problems in deriving asymptotic properties
All properties of the AR(1) can be generalized to the AR(p):
yt =a0 +a1yt−1 +···+apyt−p +ut.
Eric Eisenstat (School of Economics) ECON3350/7350 Week 2 9 / 18
Stationarity
There are several forms of stationarity that can be used to describe a stochastic process. We will keep things simple with the following definition.
Definition
A stochastic process is stationary if the data generating process is such that the mean, variance, and covariances are independent of time, i.e.
E(yt) = μ, Var(yt) = σy2 = γ0,
Cov(yt, yt−k) = γk, k ≥ 1,
hold for all t.
For now, we will only consider stationary processes, and we will relax this
assumption later in the course.
Eric Eisenstat (School of Economics) ECON3350/7350 Week 2 10 / 18
Moving Average Models
Whatiftheerrorsu1,...,uT arealsocorrelated? Correlated errors could be modeled, for example, by
ut = εt + β1εt−1,
where εt is the uncorrelated innovation in the process.
The above is called a first-order moving average MA(1) process for ut. Can be generalized to MA(q):
ut =εt +β1εt−1 +···+βqεt−q.
Eric Eisenstat (School of Economics) ECON3350/7350 Week 2 11 / 18
The Autoregressive Moving Average Model
Putting the AR(1) and and MA(1) together, we get: yt =a0 +a1yt−1 +β1εt−1 +εt,
where {εt} are uncorrelated.
This is called the autoregressive moving average model ARMA(1, 1).
Things to note:
β1 = 0 implies an AR(1) process for yt;
a1 = 0 implies a MA(1) process for yt;
if |a1| < 1, then there exists an equivalent pure MA process with infinitely many lags;
if |β1| < 1, then there exists an equivalent pure AR process with infinitely many lags.
Eric Eisenstat (School of Economics) ECON3350/7350 Week 2
12 / 18
The ARMA(p, q)
In general, we work with a model containing p autoregressive lags and q moving average lags, i.e.
pq
yt = a0 + aj yt−j + εt + βj εt−j .
j=1 j=1
This turns out to be an extremely flexible way to model any stationary time series.
Unless q = 0, we cannot use OLS to estimate all parameters. Instead, numeric optimisation is required, but fortunately, STATA (and many other packages) have built-in routines to do this!
Eric Eisenstat (School of Economics) ECON3350/7350 Week 2 13 / 18
Autocovariance and Autocorrelation Functions
Most interesting properties of a time-series can be described by its autocovariance function or autocorrelation function (ACF).
An autocorrelation is just the autocovariance scaled by the process variance, i.e. ρk = γk .
γ0
The scaling eliminates dependence on the unit of measurement (e.g.
γk is higher for a process measured in cents than the same process measures in dollars, but ρk is the same).
In general, ρk ∈ (−1, 1).
The ACF is the plot of ρk against k = 1, 2, . . . For a stationary process, ρk = ρ−k, and therefore, the ACF describes all the autocorrelations in the process.
Given data that are a realization of a stochastic process, we can estimate the underlying ACF with sample autocorrelations to compute the sample autocorrelation function (SACF) or correlogram.
Eric Eisenstat (School of Economics) ECON3350/7350 Week 2 14 / 18
Partial Autocorrelation Function (PACF)
Another way to measure the relationship between yt and yt+k is to compute the correlation with the influence of all intermediary variables yt+1,...,yt+k−1 “filteredout.”
This is unknown as the partial autocorrelation φkk; plotting φkk against k generates the partial autocorrelation function (PACF).
Things to note: φ11 = γ1;
foranAR(p),φpp =ap andφkk =0forallk>p;
in general, the PACF can be recovered from the ACF using
Yule-Walker equations.
Eric Eisenstat (School of Economics) ECON3350/7350 Week 2 15 / 18
The ACF and PACF of an AR(2) Process
DGP: yt = 0.7yt−1 − 0.49yt−2 + εt ACF
11 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2
00 −0.2 −0.2 −0.4 −0.4 −0.6 −0.6 −0.8 −0.8
−1 −1 1 2 3 4 5 6 7 8 9 10
PACF
1 2 3 4 5 6 7 8 9 10
Eric Eisenstat (School of Economics)
ECON3350/7350
Week 2
16 / 18
The ACF and PACF of a MA(2) Process
DGP: yt = εt + 0.7εt−1 − 0.49εt−2 ACF
1 0.8 0.6 0.4 0.2 0
PACF
1 0.8 0.6 0.4 0.2 0
−0.2 −0.2 −0.4 −0.4 −0.6 −0.6 −0.8 −0.8
−1 −1 1 2 3 4 5 6 7 8 9 10
1 2 3 4 5 6 7 8 9 10
Eric Eisenstat (School of Economics)
ECON3350/7350
Week 2
17 / 18
The Sample ACF and PACF for AUS GDP per capita
Let’s transform the data by taking natural logs and first differences (Why?). Then, compute the sample ACF and PACF.
Eric Eisenstat (School of Economics) ECON3350/7350 Week 2 18 / 18
The Sample ACF and PACF for AUS GDP per capita
Let’s transform the data by taking natural logs and first differences (Why?). Then, compute the sample ACF and PACF.
ACF
11 0.8 0.8 0.6 0.6 0.4 0.4 0.2 0.2
00 −0.2 −0.2 −0.4 −0.4 −0.6 −0.6 −0.8 −0.8
−1 −1 1 2 3 4 5 6 7 8 9 10
PACF
Eric Eisenstat (School of Economics)
ECON3350/7350
Week 2
18 / 18
1 2 3 4 5 6 7 8 9 10