Aggregation of Coherent Experts’ Opinions: A Tractable Extreme-Outcomes Consistent Rule

  • Marcello Basili
  • Alain Chateauneuf


The paper defines a consensus distribution with respect to experts’ opinions using a multiple quantile utility model. We show that the Steiner point (Schneider, Isr J Math 2:241–249, 1971) is the representative consensus probability. The new rule for aggregation of experts’ opinions, which can be simply evaluated by the Shapley value, is prudential and coherent.


  1. Ayyub B (2001) A practical guide on conducting expert-opinion elicitation of probabilities and consequences for corps facilities. IWR report 01-R-01Google Scholar
  2. Basili M, Chateauneuf A (2011) Extreme events and entropy: a multiple quantile utility model. Int J Approx Reason 52:1095–1102CrossRefMathSciNetMATHGoogle Scholar
  3. Bi Y, Bell D, Wang H, Guo G, Guan J (2007) Combining multiple classifier using Dempster’s rule for text categorization. Appl Artif Intell 21:211–239CrossRefGoogle Scholar
  4. Chateauneuf A, Cornet B (2012) Computing the cost spread of a financial structure with bid/ask. Working paper, Universite Paris 1 and University of KansasGoogle Scholar
  5. Cooke R (1991) Experts in uncertainty: opinion and subjective probability in science. Oxford University Press, OxfordGoogle Scholar
  6. Cooke R, Wilson A, Toumisto J, Morales O, Tanio M, Evans J (2007) A probabilistic characterization of the relationship between fine particulate matter and mortality: elicitation of European experts. Environ Sci Toxicol 41:6598–6605Google Scholar
  7. Crès H, Gilboa I, Vieille N (2011) Aggregation of multiple prior opinions. J Econ Theory 146:2563–2582CrossRefMATHGoogle Scholar
  8. Curtwright A, Morgan M, Keith D (2008) Expert assessment of future photovoltaic technology. Environ Sci Technol 42:9031–9038CrossRefGoogle Scholar
  9. de Campos L, Huete J, Moral S (1994) Probability intervals: a tool for uncertain reasoning. Int J Uncertain Fuzziness Knowl Based Syst 2:167–196CrossRefMATHGoogle Scholar
  10. DeGroot M, Montera J (1991) Optimal linear opinion pools. Manag Sci 37:546–558CrossRefMATHGoogle Scholar
  11. Denoeux T (2008) Conjunctive and disjunctive combination of belief functions induced by non distinct bodies of evidence. Artif Intell 172:234–264CrossRefMathSciNetMATHGoogle Scholar
  12. Dubois D (2011) Fuzzy measures on finite scales as families of possibility measures. EUSFLAT-LFAGoogle Scholar
  13. Dubois D, Prade H (1994) Possibility theory and data fusion in poorly informed environments. Control Eng Pract 2:812–823CrossRefGoogle Scholar
  14. Gajdos T, Vergnaud J (2013) Decisions with conflicting and imprecise information. Soc Choice Welf 41:427–452CrossRefMathSciNetMATHGoogle Scholar
  15. Gajdos T, Hayashi T, Tallon JM, Vergnaud JC (2008) Attitude toward imprecise information. J Econ Theory 140:27–65CrossRefMathSciNetMATHGoogle Scholar
  16. Genest C, Remillard B (2006) Comments on T. Mikosh’s paper “Copulas: tales and facts”. Extremes 9:27–36CrossRefMathSciNetGoogle Scholar
  17. Genest C, McConway K, Schervish M (1986) Characterization of externally Bayesian pooling operators. Ann Stat 14:487–501CrossRefMathSciNetMATHGoogle Scholar
  18. Grabish M, Labreuche C (2005) Bi-capacities II: the Choquet integral. Fuzzy Set Syst 151: 237–259CrossRefGoogle Scholar
  19. Guidance Note for Lead Authors (2010) IPCC 5th assessment report on consistent treatment of uncertainties. Intergovernmental panel on climate change (IPCC), IPCC cross-working group meeting on consistent treatment of uncertainties, Jasper Ridge, CAGoogle Scholar
  20. Hadar J, Russell W (1969) Rules for ordering uncertain prospects. Am Econ Rev 59:25–34Google Scholar
  21. Ha-Duong M (2008) Hierarchical fusion of expert opinion in the transferable belief model, application to climate sensitivity. Int J Approx Reason 49:555–574CrossRefGoogle Scholar
  22. Hammitt J, Zhang Y (2012) Combining experts’ judgments: comparison of algorithmic methods using synthetic data. TSE working paper, pp 12–293Google Scholar
  23. IPCC (2013) In: Stocker F, Qin D, Plattner G, Tignor M, Allen S, Boschung J, Nauels A, Xia Y, Bex V, Midgley PM (eds) Climate change 2013: The physical science basis. Contribution of working group I to the 5th assessment report of the intergovernmental panel on climate change. Cambridge University Press, Cambridge/New York, NYGoogle Scholar
  24. Kriegler E, Hall J, Held H, Dawson R, Schellnhuber H (2009) Imprecise probability assessment of tipping points in the climate system. Proc Natl Acad Sci USA 106:5041–5046CrossRefGoogle Scholar
  25. Levi H, Wiener Z (1998) Stochastic dominance and prospect dominance with subjective weighting functions. J Risk Uncertain 16:147–163CrossRefGoogle Scholar
  26. Owen G (1968) Game theory. Saunders Company, PhiladelphiaMATHGoogle Scholar
  27. Plous S (1993) The psychology of judgment and decision making. McGraw-Hill, New YorkGoogle Scholar
  28. Roman H, Walker K, Walsh T, Conner L, Richmond H, Hubbell B (2008) Expert judgment assessment of the mortality impact of changes in ambient fine particulate matter in the US. Environ Sci Technol 42:2268–2274CrossRefGoogle Scholar
  29. Sandri S, Dubois D, Kalfsbeek H (1995) Corrections to elicitation, assessment, and pooling of expert judgments using possibility theory. IEEE Trans Fuzzy Syst 3. doi:10.1109/TFUZZ.1995.4819Google Scholar
  30. Savage L (1971) Elicitation of personal probabilities and expectations. J Am Stat Assoc 66:783–801CrossRefMathSciNetMATHGoogle Scholar
  31. Schneider R (1971) On Steiner points of convex bodies. Isr J Math 2:241–249CrossRefGoogle Scholar
  32. Schneider R (1993) Convex bodies: the Brunn-Minkowski theory. Cambridge University Press, CambridgeCrossRefMATHGoogle Scholar
  33. Stephanou H, Lu S (1988) Measuring consensus effectiveness by a generalized entropy criterion. IEEE Trans Pattern Anal Mach Intell 10:544–554CrossRefGoogle Scholar
  34. Stone M (1961) The opinion pool. Ann Math Stat 32:1339–1342CrossRefMATHGoogle Scholar
  35. U.S. Department of Transportation/Federal Railroad Administration (2003) Chapter 2: approach to estimation of human reliability in train control system studies. In: Human reliability analysis in support of risk assessment for positive train control. DOT/FRA/ORD- 03/15. Office of Research and Development, Washington, DCGoogle Scholar
  36. U.S. Environmental Protection Agency (2005) Guidelines for carcinogen risk assessment. EPA/630/P-03-001B. Risk Assessment Forum, Office of Research and Development, Washington, DCGoogle Scholar
  37. U.S. Environmental Protection Agency (2006) Regulatory impact analysis of the final PM national ambient air quality standards, Washington, DCGoogle Scholar
  38. U.S. Environmental Protection Agency (2007) Regulatory impact analysis of the proposed revisions to the national ambient air quality standards (NAAQS) for ground-level ozoneGoogle Scholar
  39. U.S. Environmental Protection Agency (2011) Expert elicitation task force white paper, EPA-452/R-07-008, Washington, DCGoogle Scholar
  40. U.S. Office of Management and Budget (USOMB) (2002) Guidelines for ensuring and maximizing the quality, objectivity, utility, and integrity of information disseminated by federal agencies. Fed Regist 67(36):8452–8460Google Scholar
  41. U.S. Nuclear Regulatory Commission (USNRC) (1996) Branch technical position on the use of expert elicitation in the high-level radioactive waste program. NUREG-1563. Division of Waste Management, Office of Nuclear Material Safety and Standards, Washington, DCGoogle Scholar
  42. Winkler R (1968) The consensus of subjective probability distributions. Manag Sci 15:61–75CrossRefGoogle Scholar
  43. Yu D (1997) Expert opinion elicitation process using a fuzzy probability. J Korean Nucl Soc 29: 25–34Google Scholar
  44. Zickfeld K, Morganb M, Frame D, Keithd D (2010) Expert judgments about transient climate response to alternative future trajectories of radiative forcing. Proc Natl Acad Sci USA 107:12451–12456CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.DEPS and SEMUniversity of SienaSienaItaly
  2. 2.IPAG Business School and PSE-CESUniversity of Paris-IParisFrance

Personalised recommendations