CS计算机代考程序代写 Bayesian assembly algorithm Week-12 Advanced Forecasting Methods

Week-12 Advanced Forecasting Methods

Some of the slides are adapted from the lecture notes provided by Prof. Antoine Saure and Prof. Rob Hyndman

Business Forecasting Analytics
ADM 4307 – Fall 2021

Advanced Forecasting Methods

Ahmet Kandakoglu, PhD

22 November, 2021

Outline

• Complex seasonality

• Prophet model

• Vector autoregression

• Neural network models

• Bootstrapping and bagging

ADM 4307 Business Forecasting Analytics 2Fall 2021

Complex Seasonality

• So far, we have mostly considered relatively simple seasonal patterns such as quarterly and

monthly data.

• However, higher frequency time series often exhibit more complicated seasonal patterns.

• For example, daily data may have a weekly pattern as well as an annual pattern.

• Hourly data usually has three types of seasonality: a daily pattern, a weekly pattern, and an

annual pattern.

• Even weekly data can be challenging to forecast as there are not a whole number of weeks in

a year.

Fall 2021 ADM 4307 Business Forecasting Analytics 3

Complex Seasonality

• The number of calls to a North American commercial bank per 5-minute interval between

7:00am and 9:05pm each weekday over a 33 week period.

Fall 2021 ADM 4307 Business Forecasting Analytics 4

Prophet Model

• Introduced by Facebook in 2018, originally for forecasting daily data with weekly and

yearly seasonality, plus holiday effects.

• It was later extended to cover more types of seasonal data.

• It works best with time series that have strong seasonality and several seasons of

historical data.

• Prophet is robust to missing data and shifts in the trend, and typically handles outliers

well.

ADM 4307 Business Forecasting Analytics 5Fall 2021

Prophet Model

• Prophet can be considered a nonlinear regression model:

𝑦𝑡 = 𝑔 𝑡 + 𝑠 𝑡 + ℎ 𝑡 + 𝜀𝑡

where

• 𝑔 𝑡 describes a piecewise-linear trend (or “growth term”),

• 𝑠 𝑡 describes the various seasonal patterns,

• ℎ 𝑡 captures the holiday effects, and

• 𝜀𝑡 is a white noise error term.

ADM 4307 Business Forecasting Analytics 6Fall 2021

Prophet Model

• The knots (or changepoints) for the piecewise-linear trend are automatically selected

if not explicitly specified. Optionally, a logistic function can be used to set an upper

bound on the trend.

• The seasonal component consists of Fourier terms of the relevant periods. By default,

order 10 is used for annual seasonality and order 3 is used for weekly seasonality.

• Holiday effects are added as simple dummy variables.

• The model is estimated using a Bayesian approach to allow for automatic selection of

the changepoints and other model characteristics.

ADM 4307 Business Forecasting Analytics 7Fall 2021

Example: Quarterly Cement Production

library(fable.prophet)

## Create the data

cement <- aus_production %>% filter(year(Quarter) >= 1988)

## Create the train data

train <- cement %>% filter(year(Quarter) <= 2007) ## Fit models fit <- train %>%

model(

arima = ARIMA(Cement),

ets = ETS(Cement),

prophet = prophet(Cement ~ season(period = 4, order = 2, type = “multiplicative”))

)

ADM 4307 Business Forecasting Analytics 8Fall 2021

Period is specified for quarterly

data

Example: Quarterly Cement Production

fc <- fit %>% forecast(h = “2 years 6 months”)

fc %>% autoplot(cement)

ADM 4307 Business Forecasting Analytics 9Fall 2021

Example: Quarterly Cement Production

fc %>% accuracy(cement)

The Prophet forecasts are worse than either the ETS or ARIMA forecasts.

ADM 4307 Business Forecasting Analytics 10Fall 2021

# A tibble: 3 x 10

.model .type ME RMSE MAE MPE MAPE MASE RMSSE ACF1

1 arima Test -161. 216. 186. -7.71 8.68 1.27 1.26 0.387

2 ets Test -171. 222. 191. -8.07 8.85 1.30 1.29 0.579

3 prophet Test -176. 248. 216. -8.37 9.90 1.47 1.44 0.701

Outline

• Complex seasonality

• Prophet model

• Vector autoregression

• Neural network models

• Bootstrapping and bagging

ADM 4307 Business Forecasting Analytics 11Fall 2021

Artificial Neural Networks

• The study of artificial neural networks was inspired by attempts to model the

way human brain works

• Human brain primarily consists of nerve cells called neurons, linked together

with other neurons via strands of fiber called axons

• Neurologists have discovered that the human brain learns by changing the

strength of the connection of neurons upon repeated stimulation by the same

impulse

ADM 4307 Business Forecasting Analytics 12Fall 2021

Artificial Neural Networks

• Analogous to human brain structure, an ANN is composed of an inter

connected assembly of nodes and directed links

• The nodes in an ANN are often called neurons or units

• Each link is associated with a real valued weight parameter to emulate the

connection strength between neurons

• A neural network can be thought of as a network of “neurons” which are

organized in layers.

• Every neuron in the network is connected to every neuron in adjacent layers.

ADM 4307 Business Forecasting Analytics 13Fall 2021

Artificial Neural Networks

• The simplest networks contain no hidden layers and are equivalent to linear

regressions.

• Coefficients attached to the predictors are called “weights”.

• Forecasts are obtained by a linear combination of the inputs.

• Weights selected using a “learning algorithm” that minimizes a “cost function” such as

the MSE.
ADM 4307 Business Forecasting Analytics 14Fall 2021

Artificial Neural Networks

• Once we add an intermediate layer with hidden neurons, the neural network becomes

non-linear.

• A multilayer feed-forward network where each layer of nodes receives inputs from the

previous layers.

• Inputs to each node combined using linear combination.

• Result modified by nonlinear function before being output.
ADM 4307 Business Forecasting Analytics 15Fall 2021

Artificial Neural Networks

• Inputs to hidden neuron j linearly combined:

• Modified using activation function such as a sigmoid:

to give the input for the next layer.

• This tends to reduce the effect of extreme input values, thus making the network

somewhat robust to outliers.

ADM 4307 Business Forecasting Analytics 16Fall 2021

Types of Activation Functions

• Various types of activation functions (f) used

• Sigmoid

• Tanh (hyperbolic tangent)

• Sign (signum)

• Rectified Linear Unit (ReLU)

ADM 4307 Business Forecasting Analytics 17Fall 2021

𝑦 = 𝑓 𝑥

𝑥 =෍

𝑖

𝑛

𝑤𝑖𝑎𝑖 + 𝑏

Artificial Neural Networks

• Weights take random values to begin with, which are then updated using the

observed data.

• There is an element of randomness in the predictions. So the network is usually

trained several times using different random starting points, and the results are

averaged.

• Number of hidden layers, and the number of nodes in each hidden layer, must be

specified in advance.

ADM 4307 Business Forecasting Analytics 18Fall 2021

Steps involved in designing ANN

• Determine the number of nodes in input layer

• Determine the number of nodes in output layer

• Select appropriate network topology (number of hidden layers and hidden

nodes, feed-forward or recurrent architecture)

• Initialize the weights and thresholds

• Remove training examples with missing values or replace them with their most

likely values

• Train the neural network by adjusting the weights of the links until the outputs

produced are consistent with class labels of training data

ADM 4307 Business Forecasting Analytics 19Fall 2021

Neural Network Autoregression (NNAR) Models

• With time series data, lagged values of the time series can be used as inputs to a

neural network, just as we used lagged values in a autoregression (AR) model.

• We call this a neural network autoregression or NNAR model.

• NNAR(p,k) to indicate there are p lagged inputs and k nodes in the hidden layer.

• For example, a NNAR(9,5) model is a neural network with the last nine observations.

• Seasonal NNAR(p,P,k)m model has inputs (𝑦𝑡−1, 𝑦𝑡−2, 𝑦𝑡−𝑝, 𝑦𝑡−𝑚, 𝑦𝑡−2𝑚, 𝑦𝑡−𝑃𝑚), k

neurons in the hidden layer and m is the number of seasons.

ADM 4307 Business Forecasting Analytics 20Fall 2021

NNAR models in R

• The NNETAR() function fits an NNAR(p,P,k) model.

• If the values of p and P are not specified, they are selected automatically.

• For non-seasonal time series, the default is the optimal number of lags (according to

the AIC) for a linear AR(p) model.

• For seasonal time series, the default values are P=1 and p is chosen from the optimal

linear model fitted to the seasonally adjusted data.

• If k is not specified, it is set to k=(p+P+1)/2 (rounded to the nearest integer).

ADM 4307 Business Forecasting Analytics 21Fall 2021

Example: Sunspots

• The surface of the sun contains magnetic regions that appear as dark spots.

• These affect the propagation of radio waves, and so telecommunication companies

like to predict sunspot activity in order to plan for any future difficulties.

• Sunspots follow a cycle of length between 9 and 14 years.

ADM 4307 Business Forecasting Analytics 22Fall 2021

Example: Sunspots

sunspots <- sunspot.year %>% as_tsibble()

sunspots %>% autoplot(value) + labs(x = “Year”, y = “Counts”, title = “Yearly sunspots”)

ADM 4307 Business Forecasting Analytics 23Fall 2021

Example: Sunspots

sunspots %>% model(NNETAR(sqrt(value))) %>% forecast(h = 30) %>%

autoplot(sunspots) + labs(x = “Year”, y = “Counts”, title = “Yearly sunspots”)

ADM 4307 Business Forecasting Analytics 24Fall 2021

Business Forecasting Analytics
ADM 4307 – Fall 2021

Advanced Forecasting Methods

ADM 4307 Business Forecasting Analytics 25Fall 2021