Skip to main content
SpringerLink
Log in

Perspectives on Expertise in the Aggregation of Judgments

  • Chapter
Expertise and Decision Support

Abstract

Various approaches exist by which the response of a number of experts (or “judges”) may be combined in order to attempt to achieve assessment superior to that which might be attained by merely accepting an individual recommendation. Such approaches have been classified, according to Ferrell (1985), into those of “mathematical,” “behavioral,” and “mixed” type. Briefly, “mathematical” approaches entail the statistical aggregation of a number of judges into a single estimate, while “behavioral” approaches allow the full interaction of group members until some form of consensus is achieved, and “mixed” type involves components of both these approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Armstrong, J. S. (1986). Research on forecasting: A quarter-century review, 1960–1984. Interfaces, 16(11), 89–109.

    Google Scholar 

  • Ashton, A. H. (1982). An empirical study of budget-related predictions of corporate executives. Journal of Accounting Research, 20(2), 440–449.

    Article  Google Scholar 

  • Ashton, A. H., & Ashton, R. H. (1985). Aggregating subjective forecasts: Some empirical results. Management Science, 31, 1499–1508.

    Google Scholar 

  • Ashton, R. H. (1986). Combining the judgments of experts: How many and which ones? Organizational Behavior and Human Decision Processes, 38, 405–414.

    Article  Google Scholar 

  • Best, R. J. (1974). An experiment in Delphi estimation in marketing decision making. Journal of Marketing Research, 11, 448–452.

    Article  Google Scholar 

  • Bender, A. D., Strack, A. E., Ebright, G. W., & von Haunalter, G. (1969). Delphi study examines developments in medicine. Futures, 1, 289–303.

    Article  Google Scholar 

  • Brockhoff, K. (1975). The performance of forecasting groups in computer dialogue and face to face discussion. In H. Linstone & M. Turoff (Eds.), The Delphi Method: Techniques and applications. London: Addison-Wesley.

    Google Scholar 

  • Bunn, D. (1989). Forecasting with more than one model. Journal of Forecasting, 8(3), 161–166.

    Article  Google Scholar 

  • Clemen, R. T. (1989). Combining forecasts: A review and annotated bibliography. International Journal of Forecasting, 5, 559–583.

    Article  Google Scholar 

  • Dalkey, N. C., & Brown, B. (1971). Comparison of group judgment techniques with short-range predictions and almanac questions. The RAND Corporation, R-678-ARPA.

    Google Scholar 

  • Dalkey, N. C., Brown, B., & Cochran, S. W. (1970). The Delphi Method III: Use of self-ratings to improve group estimates. Technological Forecasting, 1, 283–291.

    Article  Google Scholar 

  • Dalkey, N. C., & Helmer, O. (1963). An experimental application of the Delphi Method to the use of experts. Management Science, 9, 458–467.

    Google Scholar 

  • Dawes, R. (1988). Rational choice in an uncertain world. San Diego: Harcourt Brace, Jovanovich.

    Google Scholar 

  • Pawes, R. M., & Corrigan, B. (1974). Linear models in decision making. Psychological Bulletin, 81, 95–106.

    Article  Google Scholar 

  • DeGroot, M. H. (1974). Reaching a consensus. Journal of the American Statistical Association, 69, 118–121.

    Article  Google Scholar 

  • Delbecq, A. L., Van de Ven, A. H., & Gustafson, D. H. (1975). Group techniques for program planning. Glenview, 111.: Scott Foresman.

    Google Scholar 

  • Deutsch, M., & Gerard, H. B. (1955). A study of normative and informational social influences upon individual judgment. Journal of Abnormal and Social Psychology, 51, 629–636.

    Article  Google Scholar 

  • Eils, L. C., & John, R. S. (1980). A criterion validation of multiattribute utility analysis and of group communication strategy. Organizational Behavior and Human Performance, 25, 268–288.

    Article  Google Scholar 

  • Einhorn, H. J. (1974). Expert judgment: some necessary conditions and an example. Journal of Applied Psychology, 59 (5), 562–571.

    Article  Google Scholar 

  • Einhorn, H. J., & Hogarth, R. M. (1975). Unit weighting schemes for decision making. Organizational Behavior and Human Performance, 13, 171–192.

    Article  Google Scholar 

  • Einhorn, H. J., Hogarth, R. M., & Klempner, E. (1977). Quality of group judgment. Psychological Bulletin, 84, 158–172.

    Article  Google Scholar 

  • Ferrell, W. R. (1985). Combining individual judgments. In G. Wright (Ed.), Behavioral decision making. New York: Plenum.

    Google Scholar 

  • Fischer, G. W. (1981). When oracles fail—A comparison of four procedures for aggregating subjective probability forecasts. Organizational Behavior and Human Performance, 28, 96–110.

    Article  Google Scholar 

  • Flores, B. E., & White, E. M. (1989). Subjective vs objective combining of forecasts: An experiment. Journal of Forecasting, 8, 331–341.

    Article  Google Scholar 

  • Fraser, C., & Foster, D. (1984). Social groups, nonsense groups and group polarization. In H. Tajfel (Ed), Group Processes, New York: Academic Press.

    Google Scholar 

  • Granger, C. W. J., & Newbold, P. (1975). Economic forecasting: The atheist’s viewpoint. In Renton, G. A. (Ed.), Modelling the economy. London: Heineman.

    Google Scholar 

  • Gustafson, D. H., Shukla, R. K., Delbecq, A., & Walster, G. W. (1973). A comparison study of differences in subjective likelihood estimates made by individuals, interacting groups, Delphi groups and nominal groups. Organizational Behavior and Human Performance, 9, 280–291.

    Article  Google Scholar 

  • Hackman, J. R., & Morris, C. G. (1975). Group tasks, group interaction process and group performance effectiveness: A review and proposed integration. Advances in Experimental Social Psychology, 8, 45–99.

    Article  Google Scholar 

  • Hall, J., & Watson, W. H. (1971). The effects of a normative intervention on group decision-making performance. Human Relations, 23, 299–317.

    Article  Google Scholar 

  • Hastie, R. (1986). Experimental evidence on group accuracy. In B. Grafman & G. Owen (Eds.), Decision research (Vol. 2), Greenwich, CT: JAI Press.

    Google Scholar 

  • Hill, G. W. (1982). Group versus individual performance: Are N+l heads better than one? Psychological Bulletin, 91(3), 517–539.

    Article  Google Scholar 

  • Hill, K. Q., & Fowles, J. (1975). The methodological worth of the Delphi forecasting technique. Technological Forecasting and Social Change, 7, 179–192.

    Article  Google Scholar 

  • Hoffman, L. R. (1965). Group problem solving. In L. Berkowitz (Ed.), Advances in experimental social psychology (Vol. 2). New York: Academic Press.

    Google Scholar 

  • Hogarth, R. M. (1978). A note on aggregating opinions. Organizational Behavior and Human Performance, 21, 40–46.

    Article  Google Scholar 

  • Hogarth, R. M., & Makridakis, S. (1981). Forecasting and planning: An evaluation. Management Science, 27, 115–138.

    Article  Google Scholar 

  • Janis, I. (1972). Victims of groupthink, Boston: Houghton Mifflin.

    Google Scholar 

  • Jenkins, G. M. (1974). Discussion of a paper by Newbold and Granger. Journal of the Royal Statistical Society A, 137, 148–150.

    Google Scholar 

  • Jolson, M. A., & Rossow, G. (1971). The Delphi process in marketing decision making. Journal of Marketing Research, 8, 443–448.

    Article  Google Scholar 

  • Kleinmuntz, B. (1989). Why we still use our heads instead of formulas: Towards an integrative approach. Psychological Bulletin, in press.

    Google Scholar 

  • Larreche, J. C., & Moinpour, R. (1983). Managerial judgment in marketing: The concept of expertise. Journal of Marketing Research, 20, 110–121.

    Article  Google Scholar 

  • Libby, R., & Blashfield, R. K. (1978). Performance of a compolite as a function of the number of judges. Organizational Behavior and Human Performance, 21, 121–129.

    Article  Google Scholar 

  • Lichtenstein, S., Fischhoff, B., & Phillips, L. D. (1982). Calibration of probabilities: The state of the art to 1980. In Kahneman, D., Slovic, P., & Tversky, A. (Eds.), Judgment under uncertainty: Heuristics and biases, New York: Cambridge University Press.

    Google Scholar 

  • Linstone, H. A. (1978). The Delphi Technique. In R. B. Fowles (Ed.), Handbook of futures research, Westport, CT: Greenwood Press.

    Google Scholar 

  • Linstone, H., & Turoff, M. (1975). The Delphi Method: Techniques and applications, London: Addison-Wesley.

    Google Scholar 

  • Lock, A. (1987). Integrating group judgments in subjective forecasts. In G. Wright & P. Ayton (Eds.), Judgmental forecasting, Chichester: Wiley.

    Google Scholar 

  • Lorge, I., Fox, D., Davitz, J., & Brenner, M. (1958). A survey of studies contrasting the quality of group performance and individual performance. Psychological Bulletin, 55, 337–372.

    Article  PubMed  Google Scholar 

  • McKinnon, W. J. (1966). Development of the SPAN technique for making decisions in human groups. American Behavioral Scientist, 9, 9–13.

    Article  Google Scholar 

  • Martino, J. (1983). Technological forecasting for decision-makers (2nd ed.). New York: Elsevier.

    Google Scholar 

  • Nisbett, R., & Ross, R. L. (1980). Human inference: Strategies and shortcomings of social judgment. Englewood Cliffs, N.J.: Prentice-Hall.

    Google Scholar 

  • Northcroft, M. A., & Neale, G. B. (1987). Experts, amateurs and real-estate: An anchoring and adjust perspective in property pricing decisions. Organizational Behavior and Human Decision Processes, 39, 84–97.

    Article  Google Scholar 

  • Parente, F. J., & Anderson-Parente, J. K. (1987). Delphi inquiry systems. In G. Wright & P. Ayton (Eds.), Judgmental forecasting, Chichester: Wiley.

    Google Scholar 

  • Riggs, W. E. (1983). The Delphi Method: An experimental evaluation. Technological Forecasting and Social Change, 23, 89–94.

    Article  Google Scholar 

  • Rohrbaugh, J. (1979). Improving the quality of group judgment: Social judgment analysis and the Delphi technique. Organizational Behavior and Human Performance, 24, 73–92.

    Article  Google Scholar 

  • Rowse, G. L., Gustafson, D. H., & Ludke, R. L. (1974). Comparison of rules of aggregating subjective likelihood ratios. Organizational Behavior and Human Performance, 12, 274–285.

    Article  Google Scholar 

  • Salancik, J. R., Wenger, W., & Heifer, E. (1971). The construction of delphi event statements. Technological Forecasting and Social Change, 3, 65–73.

    Article  Google Scholar 

  • Seaver, D. A. (1979). Assessing probability with multiple individuals, Unpublished doctoral dissertation, University of Southern California, Los Angeles.

    Google Scholar 

  • Sniezek, J. A., & Henry, R. A. (1989). Accuracy and confidence in group judgment. Organizational Behavior and Human Decision Processes, 43, 1–28.

    Article  Google Scholar 

  • Steiner, I. D. (1972). Group process and productivity. New York: Academic Press.

    Google Scholar 

  • Tuckman, J., & Lorge, I. (1962). Individual ability as a determinant of group superiority. Human Relations, 15, 45–51.

    Article  Google Scholar 

  • Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185, 1124–1131.

    Article  PubMed  Google Scholar 

  • Uecker, W. L. (1982). The quality of group performance in simplified information evaluation. Journal of Accounting Research, 20, 388–402.

    Article  Google Scholar 

  • Vande Ven, A. H., & Delbecq, A. L. (1971). Nominal versus interacting group processes for committee decision making effectiveness. Academic Management Journal, 14, 203–213.

    Article  Google Scholar 

  • Winkler, R. L. (1971). Probablistic prediction: Some experimental results. Journal of the American Statistical Association, 66, 675–685.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Bristol Business School, Coldharbour Lane, Frenchay, Bristol, BS16 1QY, England

    Gene Rowe

Authors
  1. Gene Rowe
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. University of Strathclyde Glasgow, Scotland

    George Wright

  2. University College London London, England

    Fergus Bolger

Rights and permissions

Reprints and permissions

Copyright information

© 1992 Plenum Press, New York

About this chapter

Cite this chapter

Rowe, G. (1992). Perspectives on Expertise in the Aggregation of Judgments. In: Wright, G., Bolger, F. (eds) Expertise and Decision Support. Springer, Boston, MA. https://doi.org/10.1007/978-0-585-34290-0_8

Download citation

  • DOI: https://doi.org/10.1007/978-0-585-34290-0_8

  • Publisher Name: Springer, Boston, MA

  • Print ISBN: 978-0-306-43862-2

  • Online ISBN: 978-0-585-34290-0

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation