Week-11 Judgmental Forecasts
Some of the slides are adapted from the lecture notes provided by Prof. Antoine Saure and Prof. Rob Hyndman
Business Forecasting Analytics
ADM 4307 – Fall 2021
Judgmental Forecasting
Ahmet Kandakoglu, PhD
22 November, 2021
Outline
• Judgmental Forecasting
• Biases and Limitations
• Key Principles
• Judgmental Methods
• Executive opinions
• Sales force opinions
• Consumer surveys
• Historical analogies
• Expert opinions
• Scenario-based forecasting
• Judgmental Adjustments
ADM 4307 Business Forecasting Analytics 2Fall 2021
Approaches to Forecasting
• Judgmental (Qualitative)
• Non-quantitative analysis of subjective inputs
• Considers “soft” information such as human factors, experience, etc.
• Quantitative: analyze “hard” data
• Time series models
• Extends historical patterns of numerical data
• Associative models
• Create equations with explanatory variables to predict the future
ADM 4307 Business Forecasting Analytics 3Fall 2021
Judgmental Forecasting
• Forecasting using judgement is common in practice.
• In many cases, judgmental forecasting is the only option, such as
• when there is a complete lack of historical data, or
• when a new product is being launched, or
• when a new competitor enters the market, or
• during completely new and unique market conditions, or etc.
ADM 4307 Business Forecasting Analytics 4Fall 2021
Judgmental Forecasting
• There are three general settings in which judgmental forecasting is used:
• there are no available data, so that statistical methods are not applicable and
judgmental forecasting is the only feasible approach;
• data are available, statistical forecasts are generated, and these are then
adjusted using judgement; and
• data are available and statistical and judgmental forecasts are generated
independently and then combined.
• We should clarify that when data are available, applying statistical methods is
preferable and should always be used as a starting point.
ADM 4307 Business Forecasting Analytics 5Fall 2021
Biases and Limitations
• It is important to recognize that judgmental forecasting is subjective, and comes with
biases and limitations:
• Judgmental forecasts can be inconsistent
• Anchoring that may lead to conservatism and undervaluing new and more current
information
• Recency that allows the most recent events dominate those in the less recent
past, which are downgraded or ignored
• Relying only the complete available information
• Judgement can be clouded by personal or political agendas, where targets and
forecasts are not segregated
• Underestimating uncertainty
• …
ADM 4307 Business Forecasting Analytics 6Fall 2021
Key Principles
• Using a systematic and well structured approach in judgmental forecasting helps to
reduce the adverse effects of the bias and limitations of judgmental forecasting.
• The following principles should be followed:
• Set the forecasting task clearly and concisely
• Implement a systematic approach to improve accuracy and consistency
• Document the decision rules and assumptions and justify
• Systematically monitor the forecasting process can identify unforeseen
irregularities
• Segregate forecasters and users
ADM 4307 Business Forecasting Analytics 7Fall 2021
Judgmental Methods
• Executive opinions
• Pool opinions of high-level executives
• Long term strategic or new product development
• Sales force opinions
• Based on direct customer contact
• Consumer surveys
• Questionnaires or focus groups
• Historical analogies
• Use demand for a similar product
• Expert opinions
• Delphi method: iterative questionnaires circulated until consensus is reached
• Scenario-based forecasting
ADM 4307 Business Forecasting Analytics 8Fall 2021
The Delphi Method
• The aim of the Delphi method is to construct consensus forecasts from a group of
experts in a structured iterative manner.
ADM 4307 Business Forecasting Analytics 9Fall 2021
• It involves the following stages:
• A panel of experts is assembled.
• Forecasting tasks/challenges are set and
distributed to the experts.
• Experts return initial forecasts and justifications.
These are compiled and summarized in order to
provide feedback.
• Feedback is provided to the experts, who now
review their forecasts in light of the feedback.
This step may be iterated until a satisfactory
level of consensus is reached.
• Final forecasts are constructed by aggregating
the experts’ forecasts.
Questionnaire
Analysis
Reformulation of Questions
Synthesis
Consensus
The Delphi Method
• The method relies on the key assumption that forecasts from a group are generally
more accurate than those from individuals.
• The first challenge of the facilitator is to identify a group of experts who can contribute
to the forecasting task.
• The usual suggestion is somewhere between 5 and 20 experts with diverse
expertise.
• A key feature of the Delphi method is that the participating experts remain
anonymous at all times.
• This avoids the situation where a group meeting is held and some members do
not contribute, while others dominate.
• It also prevents members exerting undue influence based on seniority or
personality.
ADM 4307 Business Forecasting Analytics 10Fall 2021
The Delphi Method
• Feedback to the experts should include summary statistics of the forecasts and
outlines of qualitative justifications
• Numerical data summaries and graphical representations can be used to summarize
the experts’ forecasts.
• Satisfactory consensus does not mean complete convergence in the forecast value; it
simply means that the variability of the responses has decreased to a satisfactory
level.
• Applying the Delphi method can be time consuming. In a group meeting, final
forecasts can possibly be reached in hours or even minutes.
• The role of the facilitator is of the utmost importance.
ADM 4307 Business Forecasting Analytics 11Fall 2021
Historical Analogies
• A useful judgmental approach which is often implemented in practice is forecasting by
analogy.
• A common example is the pricing of a house through an appraisal process. An
appraiser estimates the market value of a house by comparing it to similar properties
that have sold in the area.
• Even thinking and discussing analogous products or situations can generate useful
(and sometimes crucial) information.
• A structured analogy approach comprising a panel of experts can be implemented.
ADM 4307 Business Forecasting Analytics 12Fall 2021
Scenario Forecasting
• The aim of this approach is to generate forecasts based on plausible scenarios.
• In contrast to the other approaches where the resulting forecast is intended to be a
likely outcome, each scenario-based forecast may have a low probability of
occurrence.
• The scenarios are generated by considering all possible factors or drivers.
• Building forecasts based on scenarios allows a wide range of possible forecasts to be
generated and some extremes to be identified.
• For example, it is usual for “best”, “middle” and “worst” case scenarios to be
presented.
ADM 4307 Business Forecasting Analytics 13Fall 2021
Sales Force Opinions
• In this approach, forecasts for each outlet/branch/store of a company are generated
by salespeople, and are then aggregated.
• This usually involves sales managers forecasting the demand for the outlet they
manage.
• Salespeople are usually closest to the interaction between customers and products,
and often develop an intuition about customer purchasing intentions.
• They bring this valuable experience and expertise to the forecast.
• However, having salespeople generate forecasts violates the key principle of
segregating forecasters and users, which can create biases in many directions.
ADM 4307 Business Forecasting Analytics 14Fall 2021
Executive Opinions
• In contrast to the sales force composite, this approach involves staff at the top of the
managerial structure generating aggregate forecasts.
• Such forecasts are usually generated in a group meeting, where executives
contribute information from their own area of the company.
• Having executives from different functional areas of the company promotes great skill
and knowledge diversity in the group.
• This process carries all of the advantages and disadvantages of a group meeting
setting.
ADM 4307 Business Forecasting Analytics 15Fall 2021
Consumer Surveys
• Customer intentions can be used to forecast the demand for a new product or for a
variation on an existing product
• Questionnaires are filled in by customers on their intentions to buy the product.
• A structured questionnaire is used, asking customers to rate the likelihood of them
purchasing the product on a scale.
• For example, highly likely, likely, possible, unlikely, highly unlikely.
• We must keep in mind the relationship between purchase intention and purchase
behavior. Customers do not always do what they say they will.
ADM 4307 Business Forecasting Analytics 16Fall 2021
Judgmental Adjustments
• The situation where historical data are available and are used to generate statistical
forecasts.
• It is common for practitioners to then apply judgmental adjustments to these
forecasts.
• These adjustments can potentially provide all of the advantages of judgmental
forecasting.
• For example, they provide an avenue for incorporating factors that may not be accounted
for in the statistical model, such as promotions, large sporting events, holidays, or recent
events that are not yet reflected in the data.
• Judgmental adjustments are most effective when there is significant additional
information at hand or strong evidence of the need for an adjustment.
ADM 4307 Business Forecasting Analytics 17Fall 2021
Business Forecasting Analytics
ADM 4307 – Fall 2021
Judgmental Forecasting
ADM 4307 Business Forecasting Analytics 18Fall 2021