Reporting probabilistic expectations with dynamic uncertainty about possible distributions


We study how experimental subjects report subjective probability distributions in the presence of ambiguity characterized by uncertainty over a fixed set of possible probability distributions generating future outcomes. Subjects observe draws from the true but unknown probability distribution generating outcomes at the beginning of each period of the experiment and state at selected periods a) the likelihoods that each probability distribution in the set is the true distribution, and b) the likelihoods of future outcomes. We estimate heterogeneity of rules used to update uncertainty about the true distribution and rules used to report distributions of future outcomes. We find that approximately 65% of subjects report distributions by properly weighing the possible distributions using their expressed uncertainty over them, while 22% of subjects report distributions close to the distribution they perceive as most likely. We find significant heterogeneity in how subjects update their expressed uncertainty. On average, subjects tend to overweigh the importance of their prior uncertainty relative to new information, leading to ambiguity that is substantially more persistent than would be predicted using Bayes’ rule. Counterfactual simulations suggest that this persistence will likely hold in settings not covered by our experiment.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7


  1. 1.

    Early surveys of subjective probability measurement include van Lenthe (1993), McClelland and Bolder (1994), and Manski (2004).

  2. 2.

    Interpreting second-order probabilities as ambiguity is common but also criticized, see Trautmann and van de Kuilen (2015a) for a discussion. Moreover, Qiu and Weitzel (2016) find the smooth model of ambiguity to perform best.

  3. 3.

    In practice each possible distribution had an equal chance of being designated the true distribution. Participants were not informed about the probability with which one of them would be drawn. This was to ensure genuine distribution uncertainty.

  4. 4.

    Graphical interfaces have been found to be effective in helping subjects report accurate probability distributions. See Goldstein and Rothschild (2014) for recent experimental evidence.

  5. 5.

    These results add to recent work suggesting that non-incentivized belief elicitation provides reliable data (see Armantier and Treich 2013 and Trautmann and van de Kuilen 2015b).

  6. 6.

    Translated written instructions are available in the Electronic Supplementary Material.

  7. 7.

    We added a treatment, where participants could hit a button “I don’t know” when they could not or did not want to express their judgement concerning the true distribution or the prediction task. As a result, this option was used only ten times (0.56%) for all elicitation periods and all subjects, and we did not find differences in responses between both treatment groups, and thus we will not discuss this treatment in great detail.

  8. 8.

    This function was initially proposed by Prelec (1998) to model how individuals overweigh or down-weigh objective probabilities, a central element of prospect theory. We use it differently than originally intended to weigh subjective probabilities that a person associates with a particular prior distribution.

  9. 9.

    We also considered a one-step approach, directly estimating (α, β, γ) by solving the following minimization problem

    $$\underset{\left( \alpha ,\beta ,\gamma \right) }{\min }\sum\limits_{t\in \mathcal{T}} \sum\limits_{k = 1}^{6}\left( \pi_{k,t}^{r}-w\left( {\mu_{t}^{j}}\right) \pi_{k}^{1}-w\left( {\mu_{t}^{j}}\right) {\pi_{k}^{2}}-w\left( \mu_{t}^{j}\right) {\pi_{k}^{3}}\right)^{2} $$

    for each subject. This one-step estimation approach turned out to be numerically unstable for many subjects.

  10. 10.

    We also estimated a specification allowing δ1 and δ2 to change over time. This proved to be numerically unstable for many subjects.

  11. 11.

    Throughout we set \({\mu ^{j}_{0}}= 0.5\) for j = 1, 2.

  12. 12.

    The analysis was conducted for the 74 subjects for whom all 5 behavioral parameters were successfully estimated.


  1. Al-Nowaihi, A., & Dhami, S. (2010). Probability weighting functions. Wiley Encyclopedia of Operations Research and Management Science.

  2. Armantier, O., & Treich, N. (2013). Eliciting beliefs: Proper scoring rule, incentives, stakes and hedging. European Economic Review, 62, 17–40.

    Article  Google Scholar 

  3. Baillon, A., Bleichrodt, H., Keskin, U., l’Haridon, O., Li, C. (2017). The effect of learning on ambiguity attitudes. Management Science, 64, 2181–2198.

    Article  Google Scholar 

  4. Bellemare, C., Kröger, S., Van Soest, A. (2008). Measuring inequity aversion in a heterogeneous population using experimental decisions and subjective probabilities. Econometrica, 76, 815–839.

    Article  Google Scholar 

  5. Bruine de Bruin, W., Fischbeck, P.S., Stiber, N.A., Fischhoff, B. (2002). What number is “fifty-fifty”?: Redistributing excessive 50% responses in elicited probabilities. Risk Analysis, 22, 713–723.

    Article  Google Scholar 

  6. Cohen, M., Gilboa, I., Jaffray, J.-Y., Schmeidler, D. (2000). An experimental study of updating ambiguous beliefs. Risk Decision and Policy, 5, 123–133.

    Article  Google Scholar 

  7. Delavande, A., & Manski, C.F. (2015). Using elicited choice probabilities in hypothetical elections to study decisions to vote. Electoral Studies, 38, 28–37.

    Article  Google Scholar 

  8. Dominiak, A., Duersch, P., Lefort, J.-P. (2012). A dynamic Ellsberg urn experiment. Games and Economic Behavior, 75, 625–638.

    Article  Google Scholar 

  9. Dominitz, J., & Manski, C.F. (2007). Expected equity returns and portfolio choice: Evidence from the health and retirement study. Journal of the European Economic Association, 5, 369–379.

    Article  Google Scholar 

  10. Dominitz, J., & Manski, C.F. (2011). Measuring and interpreting expectations of equity returns. Journal of Applied Econometrics, 26, 352–370.

    Article  Google Scholar 

  11. Edwards, W. (1982). Conservatism in human information processing (excerpted). In Judgment under uncertainty: Heuristics and biases. Cambridge: Cambridge University Press.

  12. Ellsberg, D. (1961). Risk, ambiguity, and the Savage axioms. The Quarterly Journal of Economics, 75, 643–669.

    Article  Google Scholar 

  13. Engle-Warnick, J., & Laszlo, S. (2017). Learning-by-doing in an ambiguous environment. Journal of Risk and Uncertainty, 55, 71–94.

    Article  Google Scholar 

  14. Fischbacher, U. (2007). z-Tree: Zurich toolbox for ready-made economic experiments. Experimental Economics, 10, 171–178.

    Article  Google Scholar 

  15. Goldstein, G.D., & Rothschild, D. (2014). Lay understanding of probability distributions. Judgment and Decision Making, 9, 1–14.

    Google Scholar 

  16. Grether, D.M. (1980). Bayes rule as a descriptive model: The representativeness heuristic. The Quarterly Journal of Economics, 95, 537–557.

    Article  Google Scholar 

  17. Grossman, Z., & Owens, D. (2012). An unlucky feeling: Overconfidence and noisy feedback. Journal of Economic Behavior & Organization, 84, 510–524.

    Article  Google Scholar 

  18. Kahneman, D., & Tversky, A. (1973). On the psychology of prediction. Psychological Review, 80, 237–251.

    Article  Google Scholar 

  19. Klibanoff, P., Marinacci, M., Mukerji, S. (2005). A smooth model of decision making under ambiguity. Econometrica, 73, 1849–1892.

    Article  Google Scholar 

  20. Manski, C.F. (2004). Measuring expectations. Econometrica, 72, 1329–1376.

    Article  Google Scholar 

  21. Manski, C., & Neri, C. (2013). First- and second-order subjective expectations in strategic decision-making: Experimental evidence. Games and Economic Behavior, 81, 232–254.

    Article  Google Scholar 

  22. McClelland, A., & Bolder, F. (1994). The calibration of subjective probabilities: Theories and models 1980–1994. In Subjective probability (pp. 453–481). New York: Wiley.

  23. Mobius, M.M., Niederle, M., Niehaus, P., Rosenblat, T.S. (2011). Managing self-confidence: Theory and experimental evidence, Tech. rep., National Bureau of Economic Research.

  24. Moreno, O.M., & Rosokha, Y. (2016). Learning under compound risk vs. learning under ambiguity—an experiment. Journal of Risk and Uncertainty, 53, 137–162.

    Article  Google Scholar 

  25. Neri, C. (2015). Eliciting beliefs in continuous-choice games: A double auction experiment. Experimental Economics, 18, 569–608.

    Article  Google Scholar 

  26. Offerman, T., Sonnemans, J., van de Kuilen, G., Wakker, P. (2009). A truth serum for non-Bayesians: Correcting proper scoring rules for risk attitudes. Review of Economic Studies, 76, 1461–1489.

    Article  Google Scholar 

  27. Prelec, D. (1998). The probability weighting function. Econometrica, 66, 497–527.

    Article  Google Scholar 

  28. Qiu, J., & Weitzel, U. (2016). Experimental evidence on valuation with multiple priors. Journal of Risk and Uncertainty, 53, 55–74.

    Article  Google Scholar 

  29. Rappange, D.R., Brouwer, W.B., van Exel, J. (2016). A long life in good health: Subjective expectations regarding length and future healthrelated quality of life. The European Journal of Health Economics, 17(5), 577–589.

    Article  Google Scholar 

  30. Trautmann, S., & van de Kuilen, G. (2015a). Ambiguity attitudes. In Wiley Blackwell handbook of judgment and decision making (pp. 89–116). Oxford: Blackwell.

    Google Scholar 

  31. Trautmann, S., & van de Kuilen, G. (2015b). Belief elicitation: A horse race among truth serums. Economic Journal, 125, 2116–2135.

    Article  Google Scholar 

  32. Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185, 1124–1131.

    Article  Google Scholar 

  33. van Lenthe, J. (1993). ELI: An interactive elicitation technique for subjective probability distributions. Organizational Behavior and Human Decision Processes, 55, 379–413.

    Article  Google Scholar 

  34. Wiswall, M., & Zafar, B. (2015). Determinants of college major choice: Identification using an information experiment. The Review of Economic Studies, 82, 791–824.

    Article  Google Scholar 

Download references


The authors thank Peter Wakker for very helpful comments and suggestions.

Author information



Corresponding author

Correspondence to Charles Bellemare.

Electronic supplementary material

Below is the link to the electronic supplementary material.

(PDF 306 KB)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Bellemare, C., Kröger, S. & Sossou, K.M. Reporting probabilistic expectations with dynamic uncertainty about possible distributions. J Risk Uncertain 57, 153–176 (2018).

Download citation


  • Measurement of expectations
  • Belief updating
  • Preferences under ambiguity

JEL Classifications

  • D03
  • D84
  • C50