Week-12 Advanced Forecasting Methods
Some of the slides are adapted from the lecture notes provided by Prof. Antoine Saure and Prof. Rob Hyndman
Business Forecasting Analytics
ADM 4307 – Fall 2021
Advanced Forecasting Methods
Ahmet Kandakoglu, PhD
22 November, 2021
Outline
• Complex seasonality
• Prophet model
• Vector autoregression
• Neural network models
• Bootstrapping and bagging
ADM 4307 Business Forecasting Analytics 2Fall 2021
Complex Seasonality
• So far, we have mostly considered relatively simple seasonal patterns such as quarterly and
monthly data.
• However, higher frequency time series often exhibit more complicated seasonal patterns.
• For example, daily data may have a weekly pattern as well as an annual pattern.
• Hourly data usually has three types of seasonality: a daily pattern, a weekly pattern, and an
annual pattern.
• Even weekly data can be challenging to forecast as there are not a whole number of weeks in
a year.
Fall 2021 ADM 4307 Business Forecasting Analytics 3
Complex Seasonality
• The number of calls to a North American commercial bank per 5-minute interval between
7:00am and 9:05pm each weekday over a 33 week period.
Fall 2021 ADM 4307 Business Forecasting Analytics 4
Prophet Model
• Introduced by Facebook in 2018, originally for forecasting daily data with weekly and
yearly seasonality, plus holiday effects.
• It was later extended to cover more types of seasonal data.
• It works best with time series that have strong seasonality and several seasons of
historical data.
• Prophet is robust to missing data and shifts in the trend, and typically handles outliers
well.
ADM 4307 Business Forecasting Analytics 5Fall 2021
Prophet Model
• Prophet can be considered a nonlinear regression model:
𝑦𝑡 = 𝑔 𝑡 + 𝑠 𝑡 + ℎ 𝑡 + 𝜀𝑡
where
• 𝑔 𝑡 describes a piecewise-linear trend (or “growth term”),
• 𝑠 𝑡 describes the various seasonal patterns,
• ℎ 𝑡 captures the holiday effects, and
• 𝜀𝑡 is a white noise error term.
ADM 4307 Business Forecasting Analytics 6Fall 2021
Prophet Model
• The knots (or changepoints) for the piecewise-linear trend are automatically selected
if not explicitly specified. Optionally, a logistic function can be used to set an upper
bound on the trend.
• The seasonal component consists of Fourier terms of the relevant periods. By default,
order 10 is used for annual seasonality and order 3 is used for weekly seasonality.
• Holiday effects are added as simple dummy variables.
• The model is estimated using a Bayesian approach to allow for automatic selection of
the changepoints and other model characteristics.
ADM 4307 Business Forecasting Analytics 7Fall 2021
Example: Quarterly Cement Production
library(fable.prophet)
## Create the data
cement <- aus_production %>% filter(year(Quarter) >= 1988)
## Create the train data
train <- cement %>% filter(year(Quarter) <= 2007) ## Fit models fit <- train %>%
model(
arima = ARIMA(Cement),
ets = ETS(Cement),
prophet = prophet(Cement ~ season(period = 4, order = 2, type = “multiplicative”))
)
ADM 4307 Business Forecasting Analytics 8Fall 2021
Period is specified for quarterly
data
Example: Quarterly Cement Production
fc <- fit %>% forecast(h = “2 years 6 months”)
fc %>% autoplot(cement)
ADM 4307 Business Forecasting Analytics 9Fall 2021
Example: Quarterly Cement Production
fc %>% accuracy(cement)
The Prophet forecasts are worse than either the ETS or ARIMA forecasts.
ADM 4307 Business Forecasting Analytics 10Fall 2021
# A tibble: 3 x 10
.model .type ME RMSE MAE MPE MAPE MASE RMSSE ACF1
1 arima Test -161. 216. 186. -7.71 8.68 1.27 1.26 0.387
2 ets Test -171. 222. 191. -8.07 8.85 1.30 1.29 0.579
3 prophet Test -176. 248. 216. -8.37 9.90 1.47 1.44 0.701
Outline
• Complex seasonality
• Prophet model
• Vector autoregression
• Neural network models
• Bootstrapping and bagging
ADM 4307 Business Forecasting Analytics 11Fall 2021
Artificial Neural Networks
• The study of artificial neural networks was inspired by attempts to model the
way human brain works
• Human brain primarily consists of nerve cells called neurons, linked together
with other neurons via strands of fiber called axons
• Neurologists have discovered that the human brain learns by changing the
strength of the connection of neurons upon repeated stimulation by the same
impulse
ADM 4307 Business Forecasting Analytics 12Fall 2021
Artificial Neural Networks
• Analogous to human brain structure, an ANN is composed of an inter
connected assembly of nodes and directed links
• The nodes in an ANN are often called neurons or units
• Each link is associated with a real valued weight parameter to emulate the
connection strength between neurons
• A neural network can be thought of as a network of “neurons” which are
organized in layers.
• Every neuron in the network is connected to every neuron in adjacent layers.
ADM 4307 Business Forecasting Analytics 13Fall 2021
Artificial Neural Networks
• The simplest networks contain no hidden layers and are equivalent to linear
regressions.
• Coefficients attached to the predictors are called “weights”.
• Forecasts are obtained by a linear combination of the inputs.
• Weights selected using a “learning algorithm” that minimizes a “cost function” such as
the MSE.
ADM 4307 Business Forecasting Analytics 14Fall 2021
Artificial Neural Networks
• Once we add an intermediate layer with hidden neurons, the neural network becomes
non-linear.
• A multilayer feed-forward network where each layer of nodes receives inputs from the
previous layers.
• Inputs to each node combined using linear combination.
• Result modified by nonlinear function before being output.
ADM 4307 Business Forecasting Analytics 15Fall 2021
Artificial Neural Networks
• Inputs to hidden neuron j linearly combined:
• Modified using activation function such as a sigmoid:
to give the input for the next layer.
• This tends to reduce the effect of extreme input values, thus making the network
somewhat robust to outliers.
ADM 4307 Business Forecasting Analytics 16Fall 2021
Types of Activation Functions
• Various types of activation functions (f) used
• Sigmoid
• Tanh (hyperbolic tangent)
• Sign (signum)
• Rectified Linear Unit (ReLU)
ADM 4307 Business Forecasting Analytics 17Fall 2021
𝑦 = 𝑓 𝑥
𝑥 =
𝑖
𝑛
𝑤𝑖𝑎𝑖 + 𝑏
Artificial Neural Networks
• Weights take random values to begin with, which are then updated using the
observed data.
• There is an element of randomness in the predictions. So the network is usually
trained several times using different random starting points, and the results are
averaged.
• Number of hidden layers, and the number of nodes in each hidden layer, must be
specified in advance.
ADM 4307 Business Forecasting Analytics 18Fall 2021
Steps involved in designing ANN
• Determine the number of nodes in input layer
• Determine the number of nodes in output layer
• Select appropriate network topology (number of hidden layers and hidden
nodes, feed-forward or recurrent architecture)
• Initialize the weights and thresholds
• Remove training examples with missing values or replace them with their most
likely values
• Train the neural network by adjusting the weights of the links until the outputs
produced are consistent with class labels of training data
ADM 4307 Business Forecasting Analytics 19Fall 2021
Neural Network Autoregression (NNAR) Models
• With time series data, lagged values of the time series can be used as inputs to a
neural network, just as we used lagged values in a autoregression (AR) model.
• We call this a neural network autoregression or NNAR model.
• NNAR(p,k) to indicate there are p lagged inputs and k nodes in the hidden layer.
• For example, a NNAR(9,5) model is a neural network with the last nine observations.
• Seasonal NNAR(p,P,k)m model has inputs (𝑦𝑡−1, 𝑦𝑡−2, 𝑦𝑡−𝑝, 𝑦𝑡−𝑚, 𝑦𝑡−2𝑚, 𝑦𝑡−𝑃𝑚), k
neurons in the hidden layer and m is the number of seasons.
ADM 4307 Business Forecasting Analytics 20Fall 2021
NNAR models in R
• The NNETAR() function fits an NNAR(p,P,k) model.
• If the values of p and P are not specified, they are selected automatically.
• For non-seasonal time series, the default is the optimal number of lags (according to
the AIC) for a linear AR(p) model.
• For seasonal time series, the default values are P=1 and p is chosen from the optimal
linear model fitted to the seasonally adjusted data.
• If k is not specified, it is set to k=(p+P+1)/2 (rounded to the nearest integer).
ADM 4307 Business Forecasting Analytics 21Fall 2021
Example: Sunspots
• The surface of the sun contains magnetic regions that appear as dark spots.
• These affect the propagation of radio waves, and so telecommunication companies
like to predict sunspot activity in order to plan for any future difficulties.
• Sunspots follow a cycle of length between 9 and 14 years.
ADM 4307 Business Forecasting Analytics 22Fall 2021
Example: Sunspots
sunspots <- sunspot.year %>% as_tsibble()
sunspots %>% autoplot(value) + labs(x = “Year”, y = “Counts”, title = “Yearly sunspots”)
ADM 4307 Business Forecasting Analytics 23Fall 2021
Example: Sunspots
sunspots %>% model(NNETAR(sqrt(value))) %>% forecast(h = 30) %>%
autoplot(sunspots) + labs(x = “Year”, y = “Counts”, title = “Yearly sunspots”)
ADM 4307 Business Forecasting Analytics 24Fall 2021
Business Forecasting Analytics
ADM 4307 – Fall 2021
Advanced Forecasting Methods
ADM 4307 Business Forecasting Analytics 25Fall 2021