TIME SERIES Applied Analytics: Frameworks and Methods 2
1
Outline
■ Examine time-series data
■ Explore and visualize time-series data
■ Review basic forecasting methods
■ Explore exponential smoothing
■ Examine ARIMA models
■ Construct forecasts using simple methods, exponential smoothing, and ARIMA.
2
Time Series
■ A series of data in order of time
■ A lot of data around us is measured over a span of time
– Stock prices
– Federal funds rate – Sales
– Weather
■ Time-series data is indexed on time.
3
Simplifying Assumptions
■
1. 2.
We make some simplifying assumptions about time-series
Consecutive observations are equally spaced
Apply a discrete-time index (even though this may only hold approximately)
– E.g., daily stock returns are only available for weekdays, not weekends and holidays.
4
Time Series Data
■ Indexed by time/date
■ We use special data structures – ts
– xts: matrix with a time index – zoo
■ Use of these data structures offer access to useful functions for manipulating, analyzing and visualizing time-series data.
5
Time Series Patterns
■ A common observation across time- series is that prior observations influence later observations.
■ The influence may be from the previous observation (trend), an observation that is a fixed number of periods ago (seasonality), or an observation that is not a fixed number of periods ago (cycle).
■ Influence of prior observations can be examined by looking at lagged correlations or autocorrelations.
6
Time Series Patterns ■ Trend
– A long term increase or decrease in the data. Could be linear, non-linear, and may go from an increasing trend to a decreasing trend
■ Seasonal
– Pattern repeats itself every season. Here season may be period of the year or
day of the week. Seasonality is always of a fixed or known frequency
■ Cyclic
– Data exhibit rises or falls that are not of a fixed frequency
■ On the other hand, a time-series data that does not exhibit any pattern is a Stationary Process. White Noise is the simplest Stationary Process.
7
Time Series Patterns
■ Which is the dominant time series pattern in these charts?
– Trend
– Seasonal – Cycle
8
Time Series Components
■ Time-series may have multiple components in varying strengths. To better understand the series it is useful to divide it into its components: :
– Trend components
– Seasonal component – Remainder
– yt =Tt +St +Rt
■ Seasonality and Trend Decomposition using Loess (STL) is a versatile and robust method for decomposing a time series
■ Decomposition can be used to measure the strength of trend and seasonality
9
Transformations and Adjustments ■ Calendar adjustments
– Sometimes variation seen in seasonal data may be due to a simple calendar effect such as differences in number of business days in a month.
■ Population adjustments
– A simple adjustment for population is to look at per-capita measures
■ Inflation adjustments
– Data involving money could be adjusted for changes in the time value of money
– Adjust for inflation, CPI, etc.
■ Mathematical transformations
– When data shows change in variance with time, the variance can be stabilized by a suitable transformation. A common transformation is to use log. A family of transformations that include both log and power transformations are Box-Cox transformations which depend on the parameter lambda.
10
Time Series Regression
■ Forecast time series of interest with another time-series
■ Important to check for assumptions
– Model is a reasonable approximation of reality
– The errors must
■ Have a mean of 0
■ Are not auto-correlated
■ Unrelated to predictors
– Useful for the errors to
■ Be normally distributed
■ Have constant variance (no heteroscedasticity)
11
Forecasts
■ Predictions with time-series data are called forecasts
■ Of course, not all forecasts are derived from time-series data. Judgmental forecasts are often used when
– There is no historical data
– Data is incomplete or available after a delay
■ Three approaches to incorporating judgmental forecasts
– Rely entirely on judgment when there is no available data
– When data are available, generate statistical forecasts and adjust using judgment
– When data are available, generate statistical and judgmental forecasts independently and combine the two.
12
Judgmental Forecasts
■ Delphi method
■ Forecasting by analogy
■ Scenario forecasting
■ New product forecasting: Sales force composite, executive opinion, customer intentions
13
SIMPLE FORECASTING METHODS
14
Simple Forecasting Methods ■ Average Method
– A very simple prediction, often the baseline in linear regression, is to use the average.
■ Naïve Method
– Future will be the same as the last observation
– Since this is the best prediction for a random walk, these are also called random walk forecasts
■ Seasonal Naïve Method
– Forecast is equal to the last observed value from the same season
■ Drift Method
– Allow forecasts to increase or decrease over time, where the amount of change over time (called the drift) is set to be average change seen in historical data
15
EXPONENTIAL SMOOTHING MODELS
16
Exponential Smoothing
■ Forecasts are weighted averages of past observations with the weights decaying exponentially such that recent observations get weighted more than distant observations.
17
Types
■ Simple exponential smoothing
– Forecasts are calculated using weighted averages, where the weights decrease exponentially. Most recent observations get the heaviest weight.
– Simplest of exponential smoothing methods
– Suitable for forecasting data with no clear trend or seasonal pattern.
■ Holt’s Method
– Extends simple exponential smoothing to allow the forecasting of data with a trend
■ Holt’s Method with Damping
– Forecasts generally display a constant trend (increasing or decreasing) indefinitely into the future. For this reason, a damping parameter is usually included.
18
Types
■ Holt-Winter’s seasonal method
– Extends Holt’s method to capture seasonality. Two types:
■ Additive method is used when seasonal variations are roughly constant
■ Multiplicative method is used when seasonal variations change in proportion to the level of the series
■ But, exponential smoothing methods are not limited to these. By considering variations of trend (none, additive, additive damped) and seasonal components (non, additive, multiplicative), there are nine exponential smoothing methods
■ For each of these methods, errors may be additive or multiplicative.
19
Exponential Smoothing Models: A Taxonomy
20
State Space Models
■ Each model consists of a
– measurement equation that describes the observed data and
– some state equations that describe how the unobserved components or states (level, trend, seasonal) change over time.
■ Hence, these are referred to as State Space Models
21
ARIMA
22
ARIMA
■ Exponential Smoothing and ARIMA are the two most widely used approaches to time-series forecasting and provide complementary approaches to the problem.
■ While Exponential Smoothing models are based on a description of trend and seasonality in the data, ARIMA models aim to describe autocorrelations in the data
23
Stationary Process
■ Stationary Process has constant mean, variance and covariance (between identically spaced data points) over time.
■ Stationarity is an assumption of many time-series analysis procedures.
■ However, most time-series are non-stationary
■ Unit Root Tests: Statistical Test of Stationarity designed to see if differencing is required. E.g., KPSS Test
24
Stationary Process
■ Fortunately, a non-stationary process can be transformed into a (weakly) stationary process through transformations that remove trend and stabilize variance.
– Stabilize variance: Box-Cox Transformation
– Remove seasonality and trend: Differencing
25
ARIMA
■ AutoRegressive Integrated Moving Average
■ ARIMA = AR + I + MA
26
Differencing (I)
■ Differencing can help stabilize the mean of a time series by removing changes in the level of a time series, and so eliminating trend and seasonality.
27
Autoregressive Models (AR)
■ Forecast a variable using a linear combination of past values of the variable
■ yt = c + φ1yt-1 + φ2yt-2 +φ3yt-3 + …. εt
– where εt is white noise
28
Moving Average (MA)
■ A moving average model uses past forecast errors in a regression-like model
■ yt=c+εt+θ1ε t−1+θ2εt−2+⋯+θqεt−q
– where εt is white noise
29
Non-seasonal ARIMA
■ y′t = c + φ1y′t−1 +⋯+ φpy′t−p+ θ1εt−1+⋯+ θqεt−q+ εt
– where y′t is the differenced series
■ ARIMA(p,d,q)
– p – order of autoregressive (AR) part
– d – degree of first differencing
– q – order of moving average (MA) part
30
Seasonal ARIMA
■ Non-Seasonal ARIMA + seasonal terms = Seasonal ARIMA
■ ARIMA(p,d,q)(P,D,Q)m
– (P,D,Q) is seasonal component
– m is number of periods per season
■ Where
– d = Number of lag differences
– p = Number of ordinary Autoregressive Lags
– q = Number of ordinary Moving average lags
– D = Number of seasonal differences
– P = Number of seasonal Autoregressive Lags
– Q = Number of seasonal Moving Average lags
– m = Seasonal period or number of observations per year
31
Estimation and Order Selection
■ ARIMA model is estimated using maximum likelihood estimation. Technique finds the values of the parameters which maximize the probability of obtaining the data that we have observed
■ Information Criteria – AIC
– BIC – AICc
■ Information are useful for selecting the best values of p and q but not for selecting the appropriate order of differencing (d). Differencing changes the data on which likelihood is computed making models with different d not comparable.
32
ARIMA Modeling in R
■ Automatic Process – auto.arima()
– Uses computational shortcuts, so solution may not be optimal
– Ensure more thorough examination by
■ stepwise=F
■ approximation=F
33
ARIMA Modeling in R
■ Steps to select ARIMA model
1. Plot data to identify any unusual observations
2. If necessary, transform data to stabilize variance
3. If the data are non-stationary, take first differences until the data are stationary
4. Examine ACF and PACF to decide on AR (ARIMA(p,d,0) and/or MA terms (ARIMA(0,d,q))
5. Try chosen model and use AICc to search for a better model
6. Check residuals by plotting. If they do not look like white noise, try a modified model
7. Once the residuals look like white noise, calculate forecasts
34
Advanced Time Series Models
■ Dynamic regression models
■ Hierarchical and grouped time series
■ TBATS model for complex seasonality
■ Vector autoregressions
■ Neural network autoregression
35
Summary
■ In this module we,
– Examined time-series data
– Explored and visualized time-series data
– Reviewed basic forecasting methods
– Explored exponential smoothing
– Examined ARIMA models
– Constructed forecasts using simple methods, exponential smoothing, and ARIMA.
36