Experimental guidance for eliciting beliefs with the Stochastic Becker–DeGroot–Marschak mechanism

  • Ingrid Burfurd
  • Tom Wilkening
Original Paper


We compare different implementations of the Stochastic Becker–DeGroot–Marschak (SBDM) belief elicitation mechanism, which is theoretically elegant but challenging to implement. In a first experiment, we compare three common formats of the mechanism in terms of speed and data quality. We find that all formats yield reports with similar levels of accuracy and precision, but that the instructions and reporting format adapted from Hao and Houser (J Risk Uncertain 44(2):161–180 2012) is significantly faster to implement. We use this format in a second experiment in which we vary the delivery method and quiz procedure. Dropping the pre-experiment quiz significantly compromises the accuracy of subject’s reports and leads to a dramatic spike in boundary reports. However, switching between electronic and paper-based instructions and quizzes does not affect the accuracy or precision of subjects’ reports.


Beliefs Elicitation Prediction accuracy Methodology 

JEL Classification

C91 D81 D83 

Supplementary material

40881_2018_46_MOESM1_ESM.pdf (95 kb)
Supplementary material 1 (pdf 96 KB)


  1. Becker, G. M., DeGroot, M. H., & Marschak, J. (1964). Measuring utility by a single-response sequential method. Behavioral Science, 9(3), 226–232.CrossRefGoogle Scholar
  2. Ducharme, W. M., & Donnell, M. L. (1973). Intrasubject comparison of four response modes for “subjective probability” assessment. Organizational Behavior and Human Performance, 10(1), 108–117.CrossRefGoogle Scholar
  3. Epstein, L. (1999). A definition of uncertainty aversion. The Review of Economic Studies, 66(3), 579–609.CrossRefGoogle Scholar
  4. Grether, D. M. (1992). Testing bayes rule and the representativeness heuristic: Some experimental evidence. Journal of Economic Behavior and Organization, 17(1), 31–57.CrossRefGoogle Scholar
  5. Hao, L., & Houser, D. (2012). Belief elicitation in the presence of naïve respondents: An experimental study. Journal of Risk and Uncertainty, 44(2), 161–180.CrossRefGoogle Scholar
  6. Harrison, G. W., & Rutström, E. E. (2009). Expected utility theory and prospect theory: One wedding and a decent funeral. Experimental Economics, 12(2), 133–158.CrossRefGoogle Scholar
  7. Hollard, G., Massoni, S., & Vergnaud, J.-C. (2016). In search of good probability assessors: An experimental comparison of elicitation rules for confidence judgments. Theory and Decision, 80(3), 363–387.CrossRefGoogle Scholar
  8. Holt, C. A., & Laury, S. K. (2002). Risk aversion and incentive effects. American Economic Review, 92(5), 1644–1655.CrossRefGoogle Scholar
  9. Holt, C. A., & Smith, A. M. (2009). An update on bayesian updating. Journal of Economic Behavior and Organization, 69(2), 125–134.CrossRefGoogle Scholar
  10. Holt, C. A., & Smith, A. M. (2016). Belief elicitation with a synchronized lottery choice menu that is invariant to risk attitudes. American Economic Journal Microeconomics, 8(1), 110–39.CrossRefGoogle Scholar
  11. Huck, S., & Weizsäcker, G. (2002). Do players correctly estimate what others do? Evidence of conservatism in beliefs. Journal of Economic Behavior and Organization, 47, 71–85.CrossRefGoogle Scholar
  12. Karni, E. (2009). A theory of medical decision making under uncertainty. Journal of Risk and Uncertainty, 39(1), 1–16.CrossRefGoogle Scholar
  13. Machina, M. J., & Schmeidler, D. (1992). A more robust definition of subjective probability. Econometrica, 60(4), 745–780.CrossRefGoogle Scholar
  14. Massoni, S., Gajdos, T., & Vergnaud, J.-C. (2014). Confidence measurement in the light of signal detection theory. Frontiers in Psychology, 1455(5), 1–13.Google Scholar
  15. Möbius, M. M., Niederle, M., Niehaus, P., & Rosenblat, T. (2007). Gender differences in incorporating performance feedback. draft, February.Google Scholar
  16. Möbius, M. M., Niederle, M., Niehaus, P., & Rosenblat, T. S. (2011). Managing self-confidence: Theory and experimental evidence. Technical report, National Bureau of Economic Research.Google Scholar
  17. Palfrey, T., & Wang, S. (2009). On eliciting beliefs in strategic games. Journal of Economic Behavior and Organization, 71, 98–109.CrossRefGoogle Scholar
  18. Schlag, K. H., Tremewan, J., & Van der Weele, J. J. (2013). A penny for your thoughts: A survey of methods for eliciting beliefs. Experimental Economics, 18(3), 1–34.Google Scholar
  19. Schotter, A., & Trevino, I. (2014). Belief elicitation in the laboratory. Annual Review of Economics, 6(1), 103–128.CrossRefGoogle Scholar
  20. Trautmann, S. T., & van de Kuilen, G. (2015). Belief elicitation: A horse race among truth serums. The Economic Journal, 125, 2116–2135.CrossRefGoogle Scholar

Copyright information

© Economic Science Association 2018

Authors and Affiliations

  1. 1.Department of EconomicsThe University of MelbourneMelbourneAustralia

Personalised recommendations