Skip to main content

Expert Opinions in Forecasting: The Role of the Delphi Technique

  • Chapter
Principles of Forecasting

Part of the book series: International Series in Operations Research & Management Science ((ISOR,volume 30))

Abstract

Expert opinion is often necessary in forecasting tasks because of a lack of appropriate or available information for using statistical procedures. But how does one get the best forecast from experts? One solution is to use a structured group technique, such as Delphi, for eliciting and combining expert judgments. In using the Delphi technique, one controls the exchange of information between anonymous panelists over a number of rounds (iterations), taking the average of the estimates on the final round as the group judgment. A number of principles are developed here to indicate how to conduct structured groups to obtain good expert judgments. These principles, applied to the conduct of Delphi groups, indicate how many and what type of experts to use (five to 20 experts with disparate domain knowledge); how many rounds to use (generally two or three); what type of feedback to employ (average estimates plus justifications from each expert); how to summarize the final forecast (weight all experts’ estimates equally); how to word questions (in a balanced way with succinct definitions free of emotive terms and irrelevant information); and what response modes to use (frequencies rather than probabilities or odds, with coherence checks when feasible). Delphi groups are substantially more accurate than individual experts and traditional groups and somewhat more accurate than statistical groups (which are made up of noninteracting individuals whose judgments are aggregated). Studies support the advantage of Delphi groups over traditional groups by five to one with one tie, and their advantage over statistical groups by 12 to two with two ties. We anticipate that by following these principles, forecasters may be able to use structured groups to harness effectively expert opinion.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 429.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 549.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 549.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  • Arkes, H. (2001), “Overconfidence in judgmental forecasting,” in J..S. Armstrong (ed.), Principles of Forecasting. Norwell, MA.: Kluwer Academic Publishers.

    Google Scholar 

  • Armstrong, J. S. (1985), Long Range Forecasting: From Crystal Ball to Computer,2nd ed., New York: Wiley. (Full text at http://hops.wharton.upenn.edu/forecast.)

    Google Scholar 

  • Bardecki, M.J. (1984), “Participants’ response to the Delphi method: An attitudinal perspective,” Technological Forecasting and Social Change, 25, 281–292.

    Article  Google Scholar 

  • Beach, L. R. and L. D. Phillips (1967), “Subjective probabilities inferred from estimates and bets,” Journal of Experimental Psychology, 75, 354–259.

    Article  Google Scholar 

  • Best, R. J. (1974), “An experiment in Delphi estimation in marketing decision making,” Journal of Marketing Research, 11, 448–452.

    Article  Google Scholar 

  • Boje, D. M. and J. K. Murnighan (1982), “Group confidence pressures in iterative decisions,” Management Science, 28, 1187–1196.

    Article  Google Scholar 

  • Brockhoff, K. (1975), “The performance of forecasting groups in computer dialogue and face to face discussions,” in H. Linstone and M. Turoff (eds.), The Delphi Method: Techniques and Applications. London: Addison-Wesley.

    Google Scholar 

  • Cooper, A., C. Woo and W. Dunkelberger (1988), “Entrepreneurs perceived chances of success,” Journal of Business Venturing, 3, 97–108.

    Article  Google Scholar 

  • Dalkey, N.C., B. Brown and S. W. Cochran (1970), “The Delphi Method III: Use of self- ratings to improve group estimates,” Technological Forecasting, 1, 283–291.

    Article  Google Scholar 

  • Dawes, R. M. (1982), “The robust beauty of improper linear models in decision making,” in D. Kahneman, P. Slovic and A. Tversky (eds.), Judgement Under Uncertainty: Heuristics and Biases. Cambridge: Cambridge University Press.

    Google Scholar 

  • Dietz, T. (1987), “Methods for analyzing data from Delphi panels: Some evidence from a forecasting study,” Technological Forecasting and Social Change, 31, 79–85.

    Article  Google Scholar 

  • Erffineyer, R. C., E. S. Erffineyer and I. M. Lane (1986), “The Delphi technique: An empirical evaluation of the optimal number of rounds,” Group and Organization Studies, 11, 120–128.

    Article  Google Scholar 

  • Erffineyer, R. C. and I.M. Lane (1984), “Quality and acceptance of an evaluative task: The effects of four group decision-making formats,” Group and Organization Studies, 9, 509–529.

    Article  Google Scholar 

  • Fischer, G. W. (1981), “When oracles fail-a comparison of four procedures for aggregating subjective probability forecasts,” Organizational Behavior and Human Performance, 28, 96–110.

    Article  Google Scholar 

  • Gigerenzer, G. (1994), “Why the distinction between single event probabilities and frequencies is important for psychology (and vice-versa),” in G. Wright and P. Ayton (eds.), Subjective Probability. Chichester, U.K.: Wiley.

    Google Scholar 

  • Goodwin, P. and G. Wright (1998), Decision Analysis for Management Judgment, 2nd ed. Chichester, U.K.: Wiley.

    Google Scholar 

  • Gustafson, D. H., R. K. Shukla, A. Delbecq and G. W. Walster (1973), “A comparison study of differences in subjective likelihood estimates made by individuals, interacting groups, Delphi groups and nominal groups,” Organizational Behavior and Human Performance, 9, 280–291.

    Article  Google Scholar 

  • Hauser, P.M. (1975), Social Statistics in Use. New York: Russell Sage.

    Google Scholar 

  • Hill, G. W. (1982), “Group versus individual performance: Are N+1 heads better than one?” Psychological Bulletin, 91, 517–539.

    Article  Google Scholar 

  • Hill, K. Q. and J. Fowles (1975), “The methodological worth of the Delphi forecasting technique,” Technological Forecasting and Social Change, 7, 179–192.

    Article  Google Scholar 

  • Hogarth, R. M. (1978), “A note on aggregating opinions,” Organizational Behavior and Human Performance, 21, 40–46.

    Article  Google Scholar 

  • Jolson, M. A. and G. Rossow (1971), “The Delphi process in marketing decision making,” Journal of Marketing Research, 8, 443–448.

    Article  Google Scholar 

  • Kahneman, D. and D. Lovallo (1993), “Timid choices and bold forecasts: A cognitive perspective on risk taking,” Management Science, 39, 17–31.

    Article  Google Scholar 

  • Larreché, J. C. and R. Moinpour (1983), “Managerial judgment in marketing: The concept of expertise,” Journal of Marketing Research, 20, 110–121.

    Article  Google Scholar 

  • Linstone, H. A. (1975), “Eight basic pitfalls: A checklist,” in H. Linstone and M. Turoff (eds.), The Delphi Method: Techniques and Applications. London: Addision-Wesley.

    Google Scholar 

  • Linstone, H. A. and M. Turoff (1975), The Delphi Method: Techniques and Applications. London: Addision-Wesley.

    Google Scholar 

  • Lock, A. (1987), “Integrating group judgments in subjective forecasts,” in G. Wright and P. Ayton (eds.), Judgmental Forecasting. Chichester, U.K.: Wiley.

    Google Scholar 

  • MacGregor, D. G. (2001), “Decomposition for judgmental forecasting and estimation,” in J. S. Armstrong (ed.), Principles of Forecasting. Norwell, MA.: Kluwer Academic Publishers.

    Google Scholar 

  • Martino, J. (1983), Technological Forecasting for Decision Making, ( 2nd ed. ). New York: American Elsevier.

    Google Scholar 

  • McClelland, A. G. R. and F. Bolger (1994), “The calibration of subjective probabilities: Theories and models 1980–1994,” in G. Wright and P. Ayton (eds.), Subjective Probability. Chichester, U.K.: Wiley.

    Google Scholar 

  • Meehl, P. E. (1954), Clinical Versus Statistical Prediction: A Theoretical Analysis and a Review of the Evidence. Minneapolis: University of Minnesota Press.

    Book  Google Scholar 

  • Miner, F. C. (1979), “A comparative analysis of three diverse group decision making approaches,” Academy of Management Journal, 22, 81–93.

    Article  Google Scholar 

  • Noelle-Neuman, E. (1970), “Wanted: Rules for wording structured questionnaires,” Public Opinion Quarterly, 34, 190–201.

    Google Scholar 

  • Parenté, F.J., J. K. Anderson, P. Myers and T. O’Brien (1984), “An examination of factors contributing to Delphi accuracy,” Journal of Forecasting, 3, 173–182.

    Article  Google Scholar 

  • Parenté, F. J. and J. K. Anderson-Parenté (1987), “Delphi inquiry systems,” in G. Wright and P. Ayton (eds.), Judgmental Forecasting. Chichester, U.K.: Wiley.

    Google Scholar 

  • Payne, S.L. (1951), The Art of Asking Questions. Princeton, NJ: Princeton University Press.

    Google Scholar 

  • Riggs, W. E. (1983), “The Delphi method: An experimental evaluation,” Technological Forecasting and Social Change, 23, 89–94.

    Article  Google Scholar 

  • Rohrbaugh, J. (1979), “Improving the quality of group judgment: Social judgment analysis and the Delphi technique,” Organizational Behavior and Human Performance, 24, 73–92.

    Article  Google Scholar 

  • Rowe, G. and G. Wright (1996), “The impact of task characteristics on the performance of structured group forecasting techniques,” International Journal of Forecasting, 12, 73–89.

    Article  Google Scholar 

  • Rowe, G. and G. Wright (1999), “The Delphi technique as a forecasting tool: Issues and analysis,” International Journal of Forecasting, 15, 353–375. (Commentary follows on pp. 377–381.)

    Google Scholar 

  • Rowe, G., G. Wright and F. Bolger (1991), “The Delphi technique: A reevaluation of research and theory,” Technological Forecasting and Social Change, 39, 235–251.

    Article  Google Scholar 

  • Sackman, H. (1975), Delphi Critique. Lexington, MA: Lexington Books.

    Google Scholar 

  • Salancik, J. R., W. Wenger and E. Helfer (1971), “The construction of Delphi event statements,” Technological Forecasting and Social Change, 3, 65–73.

    Article  Google Scholar 

  • Scheibe, M., M. Skutsch and J. Schofer (1975), “Experiments in Delphi methodology,” in H. Linstone and M. Turoff (eds.), The Delphi Method: Techniques and Applications. London: Addison-Wesley.

    Google Scholar 

  • Sniezek, J. A. (1989), “An examination of group process in judgmental forecasting,” International Journal of Forecasting, 5, 171–178.

    Article  Google Scholar 

  • Sniezek, J. A. (1990), “A comparison of techniques for judgmental forecasting by groups with common information,” Group and Organization Studies, 15, 5–19.

    Article  Google Scholar 

  • Sniezek, J. A. and T. Buckley (1991), “Confidence depends on level of aggregation,” Journal of Behavioral Decision Making, 4, 263–272.

    Article  Google Scholar 

  • Stewart, T.R. (1987), “The Delphi technique and judgmental forecasting,” Climatic Change, 11, 97–113.

    Article  Google Scholar 

  • Stewart, T.R. (2001), “Improving reliability in judgmental forecasts,” in J. S. Armstrong (ed.), Principles of Forecasting. Norwell, MA.: Kluwer Academic Publishers.

    Google Scholar 

  • Sudman, S. and N. Bradburn (1982), Asking Questions. San Francisco: Josey-Bass.

    Google Scholar 

  • Tversky, A. and D. Kahneman (1974), “Judgment under uncertainty: Heuristics and biases,” Science, 185, 1124–1131.

    Article  Google Scholar 

  • Tversky, A. and D. Kahneman (1981), “The framing of decisions and the psychology of choice,” Science, 211, 453–458.

    Article  Google Scholar 

  • Van de Ven, A. H. and A. L. Delbecq (1971), “Nominal versus interacting group processes for committee decision making effectiveness,” Academic Management Journal, 14, 203–213.

    Article  Google Scholar 

  • Van de Ven, A. H. and A. L. Delbecq (1974), “The effectiveness of nominal, Delphi, and interacting group decision making processes,” Academy of Management Journal, 17, 605–621.

    Article  Google Scholar 

  • Welty, G. (1974), “The necessity, sufficiency and desirability of experts as value forecasters,” in W. Leinfellner and E. Kohler (eds.), Developments in the Methodology ofSocial Science. Boston: Reidel.

    Google Scholar 

  • Wright, G. and P. Ayton (1994), Subjective Probability. Chichester, U.K.: Wiley.

    Google Scholar 

  • Wright, G., G. Rowe, F. Bolger and J. Gammack (1994), “Coherence, calibration and expertise in judgmental probability forecasting,” Organizational Behavior and Human Decision Processes, 57, 1–25.

    Article  Google Scholar 

  • Wright, G., C. Saunders and P. Ayton (1988), “The consistency, coherence and calibration of holistic, decomposed and recomposed judgmental probability forecasts,” Journal of Forecasting, 7, 185–199.

    Article  Google Scholar 

  • Wright, G. and P. Whalley (1983), “The supra-additivity of subjective probability,” in B. Stigum and F. Wenstop (eds.), Foundations of Risk and Utility Theory with Applications. Dordrecht: Reidel.

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2001 Springer Science+Business Media New York

About this chapter

Cite this chapter

Rowe, G., Wright, G. (2001). Expert Opinions in Forecasting: The Role of the Delphi Technique. In: Armstrong, J.S. (eds) Principles of Forecasting. International Series in Operations Research & Management Science, vol 30. Springer, Boston, MA. https://doi.org/10.1007/978-0-306-47630-3_7

Download citation

  • DOI: https://doi.org/10.1007/978-0-306-47630-3_7

  • Publisher Name: Springer, Boston, MA

  • Print ISBN: 978-0-7923-7401-5

  • Online ISBN: 978-0-306-47630-3

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics