Abstract
The purpose of this paper is to review models and methods for aggregating opinions. By the word “opinion” is meant the expression of a person’s belief concerning the outcome of an uncertain event, either in the form of a point estimate or a probability distribution1. For example, consider a group of businessmen meeting to decide upon a sales forecast, the evolution of certain share prices or an uncertain economic variable (e. g. U.K. oil revenues in 1984). Given the prevalence of such forecasting activity in many fields, the importance of this topic hardly needs emphasizing.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Asch, SE . Effects of group pressure upon the modification and distortion of judgments., In H.Guetzkow (Ed.),, Groups, leadership and men. Pittsburgh: Carnegie Press, 1951
Barnlund, D.C. A comparative study of individual, majority, and group judgment., Journal of Abnormal Social Psychology, 1959, 58, 55–60.
Belovicz, M.W., & Finch, F.E. A critical analysis of the “risky shift” phenomenon., Organizational Behavior and Human Performance, 1971, 6, 150–168.
Brown, R.W. Social psychology, New York: Free Press, 1965.
Brown, T.A. An experiment in probabilistic forecasting (R-944-ARPA)., Santa Monica, California: Rand Corporation, July 1973.
Carroll, J.D., & Chang, J-J. Analysis of individual differences on multidimensional scaling via an n-way generalization of “Eckart- Young” decomposition., Psychometrika, 1970, 35, 283–319.
Clark, R.D. III. Group induced shift toward risk: A critical appraisal. Psychological Bulletin, 1971, 76, 251–270.
Cochran, W.G. Sampling techniques (2nd edition)., New York: Wiley, 1963.
Condorcet, Marquis de. Essai sur l’application de l’analyse a la probability des decisions rendues a la pluralite des voix. Paris, 1785
Dalkey, N.C. The Delphi method: An experimental study of group opinion (RM-5888-PR). Santa Monica, California: Rand Corporation, June 1969.
Daley, N.C., Brown, B., & Cochran, S. The Delphi method, III. Use of self ratings to improve group estimates (RM-6115-PR). Santa Monica, California: Rand Corporation, November 1969.
Dalkey, N.C., & Helmer, O., An experimental application of the Delphi method to the use of experts., Management Science, 1963, 9, 458–467.
Davis, J.H,. Group performance, Reading, Mass.: Addison-Wesley, 1969.
Davis, J.H. Group decision and social interaction: A theory of social decision schemes. Psychological Review, 1973, 80, 97–125.
Dawes, R.M. An inequality concerning correlation of composites vs. composites of correlations., Oregon Research Institute Methodological Note, 1970, 1 (1).
Dawes, R.M., & Corrigan, B. Linear models in decision making., Psychological Bulletin, 1974, 81 95–106.
de Finetti, B. Does it make sense to speak of “good probability appraisers?” In I.J.Good (Ed.), The scientist speculates - An anthology of partly-baked ideas. London: Heineman, 1962.
de Finetti, B. Theory of Probability (Vol. 1 ). New York: Wiley, 1974.
De Groot, M.H. Reaching a consensus. Journal of the American Statistical Association, 1974, 69, 118–121.
Deutsch, M., & Gerard, H.B., A study of normative and informational social influences upon individual judgment., Journal of Abnormal and Social Psychology, 1955, 51, 629–636.
Einhorn, H.J. Expert judgment: Some necessary conditions and an example. Journal of Applied Psychology, 1974, 59, 562–571.
Einhorn, H.J., & Hogarth, R.M. Unit weighting schemes for decision making. Organizational Behavior and Human Performance, 1975, 13, 171–192.
Einhorn, H.J., Hogarth, R.M., & Klempner, E. Quality of group judgment, Unpublished manuscript, University of Chicago, 1975.
Eisenberg, E., & Gale, D. Consensus of subjective probabilities: The parimutuel method. Annals of Mathematical Statistics, 1959, 30, 165–168.
Eysenck, H.J. The validity of judgments as a function of the number of judges., Journal of Experimental Psychology, 1939, 25, 650–654.
Fishburn, PC . A comparative analysis of group decision methods., Behavioral Science, 1971, 538–544
Fishburn, P.C. Voter concordance, simple majorities, and group decision methods., Behavioral Science, 1973, 18, 364–376.
Fishburn, P.C. Single-peaked preferences and probabilities of cyclical majorities., Behavioral Science, 1974, 19, 21–27. (a)
Fishburn, P.C. Simple voting systems and majority rule. Behavioral Science, 19, 166–176. (b)
Fishburn, P.C. Aspects of one-stage voting rules., Management Science, 1974, 31, 422–427. (c)
Ghiselli, E.E. Theory of psychological measurement. New York: McGraw- Hill, 1964.
Goldberg, L.R. Man versus model of man: A rationale, plus some evidence, for a method of improving on clinical inferences. Psychological Bulletin, 1970, 73, 422–432.
Goodman, B.C. Action selection and likelihood ratio estimation by individuals and groups., Organizational Behavior and Human Performance, 1972, 7, 121–141.
Gordon, K.H. Group judgments in the field of lifted weights., Journal of Experimental Psychology, 1924, 3, 398–400.
Gough, R.C. The effect of group format on aggregate subjective probability distributions, Paper presented at the Fourth Research Conference on Subjective Probability, Utility and Decision Making, Rome, September 1973.
Gulliksen, H. Theory of mental tests. New York: Wiley, 1950.
Gustafson, D.H., Shukla, R.K., Delbecq, A., & Walster, G.W. A comparative study of differences in subjective likelihood estimates made by individuals, interacting groups, Delphi groups, and nominal groups., Organizational Behavior and Human Performance, 1973, 9, 280–291.
Hackman, J.R., & Kaplan, R. E. Interventions into group process: An approach to improving the effectiveness of groups. Decision Sciences, 1974, 5, 459–480.
Hoffman, L.R. Group problem solving., In L.Berkowitz (Ed.),, Advances in experimental social psychology (Vol. 2 ). New York: Academic Press, 1965.
Hoffman, L.R., & Maier, N.R.F. Valence in the adoption of solutions by problem-solving groups: Concept, method and results. Journal of Abnormal and Social Psychology, 1964, 69, 264–271.
Hoffman, L.R., & Maier, N.R.F. Valence in the adoption of solutions by problem-solving groups: II. Quality and Acceptance as goals of leaders and members., Journal of Personality and Social Psychology, 1967, 6, 175–182.
Hogarth, R.M. On the aggregation of opinion, Unpublished manuscript, INSEAD, Fontainebleau, France, 1974.
Hogarth, R.M. Cognitive processes and the assessment of subjective probability distributions., Journal of the American Statistical Association, 1975, 70, 271–289.
Kelley, H.H., & Thibaut, J.W. Group problem solving., In G.Lindzey & E.Aronson (Eds.), Handbook of Social Psychology (Vol. 4 ). Reading, Mass.: Addison-Wesley, 1968.
Kish, L. Survey sampling. New York: Wiley, 1965.
Klugman, S.F. Group and individual judgments for anticipated events., Journal of Social Psychology, 1947, 26, 21–33.
Knight, H.C. A comparison of the reliability of group and individual judgments, Unpublished master’s thesis, Columbia University, 1921.
Lorge, I., Fox, D., Davitz, J., & Brenner, M. A survey of studies contrasting the quality of group performance and individual performance., Psychological Bulletin, 1958, 55, 337–372.
Lorge, I., & Solomon, H., Two models of group behavior in the solution of eureka-type problems., Psychometrika, 1955, 20, 139–148.
Mackenzie, K.D. An analysis of risky shift experiments., Organizational Behavior and Human Performance, 1971, 6, 283–303.
Maier, N.R.F. Assets and liabilities in group problem solving: The need for an integrative function. Psychological Review, 1967, 74, 239–249.
Maier, N.R.F., Problem solving and creativity in individuals and groups, Belmont, California: Wadsworth, 1970.
Morris, P.A., Bayesian expert resolution. Unpublished doctoral dissertation, Department of Engineering Economic Systems, Stanford University, 1971
Morris, P.A., Decision analysis expert use. Management Science, 1974, 20, 1233–1241.
Morris, P.A., Modeling experts, Unpublished manuscript, February 1975.
Press, S.J., & Harman, A.J., Methodology for subjective assessment of technological advancement (R-1375)., Santa Monica, California: Rand Corporation, April 1975.
Preston, M.G. Note on the reliability and validity of the group judgment. Journal of Experimental Psychology, 1938, 22, 462–471.
Raiffa, H. Decision analysis: Introductory lectures on choices under uncertainy. Reading, Mass.: Addison-Wesley, 1968.
Roberts, H.V. Probabilistic prediction., Journal of the American Statistical Association, 1965, 60, 50–62.
Rotter, J.B. Generalized expectancies for internal versus external control of reinforcement. Psychological Monographs, 1966, 80 (Whole No. 609), 1–28.
Sackman, H. Delphi assessment: Expert opinion, forecasting, and group process (R-1283-PR). Santa Monica, California: Rand Corporation, April 1974.
Sanders, F., On subjective probability forecasting., Journal of Applied Meteorology, 1963, 2, 191–201.
Sanders, F. Skill in forecasting daily temperature and precipitation: Some experimental results. Bulletin of the American Meteorological Society, 1973, 54, 1171–1179.
Savage, L.J. The foundations of statistics, New York: Wiley, 1954.
Savage, L.J. Elicitation of personal probabilities and expectations. Journal of the American Statistical Association, 1971, 66, 783–801.
Sommer, R. Small group ecology., Psychological Bulletin, 1967, 67, 145–152.
Spetzler, C.S., & Staél von Holstein, C.-A.S. Probability encoding in decision analysis., Management Science, 1975, 22, 340–358.
Staél von Hoistein, C.-A.S. Assessment and evaluation of subjective probability distributions. Stockholm: The Economic Research Institute at the Stockholm School of Economics, 1970.
Steiner, I.D. Models for inferring relationships between group size and potential group productivity., Behavioral Science, 1966, 11, 273–283.
Steiner, I.D., & Rajaratnam, N., A model for the comparison of individual and group performance scores., Behavioral Science, 1961, 6, 142–147.
Stone, M. The opinion pool., Annals of Mathematical Statistics, 1961, 32, 1339–1342.
Stroop, J.B. Is the judgment of the group better than that of the average member of the group? Journal of Experimental Psychology, 1932, 15, 550–560.
Thomas, E.J., & Fink, C.F., Effects of group size. Psychological Bulletin, 1963, 60, 371–384.
Thorndike, R.L. The effect of discussion upon the correctness of group decisions, when the factor of majority influence is allowed for. Journal of Social Psycholgy, 1938, 9, 343–363.
Toda, M. Information receiving behavior of man., Psychological Review, 1956, 63, 204–212
Torrance, E.P. Some consequences of power differences on decision making in permanent and temporary three-man groups., In A.P.Hare, E.F.Borgatta, & R.F.Bales (Eds.), Small groups: Studies in social interaction. New York: Knopf, 1955.
Triandis, H.C. Cognitive similarity and communication in a dyad., Human Relations, 1960, 13, 175–183. (a)
Triandis, H.C. Some determinants of interpersonal communication., Human Relations, 1960, 13, 279–287. (b)
Vroom, V.H., Grant, L.D., & Cotton, T.S., The consequences of social interaction in group problem solving., Organizational Behavior and Human Performance, 1969, 4, 77–95.
Vroom, V.H., & Jago, A.G. Decision making as a social process: Normative and descriptive models of leader behavior. Decision Sciences, 1974, 5, 7430–769
Vroom, V.H., & Yetton, P.W. Leadership and decision making., Pittsburgh: University of Pittsburgh Press, 1973.
Wilson, R. The theory of syndicates., Econometrica, 1968, 36, 119–132.
Winkler, R.L., The consensus of subjective probability distributions., Management Science, 1968, 15, B-61-75.
Winkler, R.L., Scoring rules and the evaluation of probability assessors., Journal of the American Statistical Association, 1969, 64, 1073–1078.
Winkler, R.L. Probabilistic prediction: Some experimental results. 66, 675–685.
Winkler, R.L., & Cummings, L.L., On the choice of a consensus distribution in Bayesian analysis., Organizational Behavior and Human Performance, 1972, 7, 63–76.
Winkler, R.L., & Murphy, A.H., “Good” probability assessors., Journal of Applied Meteorology, 1968, 7, 751–758.
Winkler, R.L., & Murphy, A.H. Experiments in the laboratory and the real world., Organizational Behavior and Human Performance, 1973, 10, 252–270.
Zajonc, R.B. A note on group judgments and group size., Human Relations, 1962, 15, 177–180.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1977 D. Reidel Publishing Company, Dordrecht-Holland
About this chapter
Cite this chapter
Hogarth, R.M. (1977). Methods for Aggregating Opinions. In: Jungermann, H., De Zeeuw, G. (eds) Decision Making and Change in Human Affairs. Theory and Decision Library, vol 16. Springer, Dordrecht. https://doi.org/10.1007/978-94-010-1276-8_16
Download citation
DOI: https://doi.org/10.1007/978-94-010-1276-8_16
Publisher Name: Springer, Dordrecht
Print ISBN: 978-94-010-1278-2
Online ISBN: 978-94-010-1276-8
eBook Packages: Springer Book Archive