Journal of Risk and Uncertainty

, Volume 45, Issue 2, pp 159–190 | Cite as

Experts in experiments

How selection matters for estimated distributions of risk preferences
  • Hans-Martin von Gaudecker
  • Arthur van Soest
  • Erik WengströmEmail author


An ever increasing number of experiments attempts to elicit risk preferences of a population of interest with the aim of calibrating parameters used in economic models. We are concerned with two types of selection effects, which may affect the external validity of standard experiments: Sampling from a narrowly defined population of students (“experimenter-induced selection”) and self-selection due to non-response or incomplete response of participants in a random sample from a broad population. We find that both types of selection lead to a sample of experts: Participants perform significantly better than the general population, in the sense of fewer violations of revealed preference conditions. Self-selection within a broad population does not seem to matter for average preferences. In contrast, sampling from a student population leads to lower estimates of average risk aversion and loss aversion parameters. Furthermore, it dramatically reduces the amount of heterogeneity in all parameters.


Risk aversion Loss aversion Internet surveys Laboratory experiments 

JEL Classification

C90 D81 



Financial support from the Dutch Science Foundation (NWO), the Swedish Institute for Banking Research (Bankforskningsinstitutet), the Wallander-Hedelius Foundation and from the European Union under grant HPRN-CT-2002-00235 (RTN-AGE) is gratefully acknowledged. This paper has made use of the DNB Household Survey and other data collected in the CentERpanel. We thank the team of CentERdata, especially Marika Puumala, for their support with the experiments, as well as Morten Lau and Joachim Winter for very helpful comments on the experimental design. The analysis benefitted from comments received at presentations in Mannheim, Copenhagen, Gothenburg, Montreal, Berlin, Aix-en-Provence, Toulouse, at the XIIth FUR conference at LUISS in Rome, the ESA meetings in Nottingham and Tucson, and a Cemmap workshop in London. Daniel Kemptner provided able research assistance. The computation of results has been facilitated by the use of the bwGRiD (2007–2011). This paper updates and extends results from von Gaudecker et al. (2008) and an early working paper version of von Gaudecker et al. (2011).


  1. Alessie, R., Hochgürtel, S., van Soest, A. (2006). Non-take-up of tax-favored savings plans: evidence from Dutch employees. Journal of Economic Psychology, 27(4), 483–501.CrossRefGoogle Scholar
  2. Andersen, S., Harrison, G.W., Lau, M.I., Rutström, E.E. (2006). Elicitation using multiple price list formats. Experimental Economics, 9(4), 383–405.CrossRefGoogle Scholar
  3. Andersen, S., Harrison, G.W., Lau, M.I., Rutström, E.E. (2008). Eliciting risk and time preferences. Econometrica, 76(3), 583–618.CrossRefGoogle Scholar
  4. Andersen, S., Harrison, G.W., Lau, M.I., Rutström, E.E. (2010). Preference heterogeneity in experiments: comparing the field and laboratory. Journal of Economic Behavior & Organization, 73(2), 209–224.CrossRefGoogle Scholar
  5. Anderson, L., & Mellor, J. (2009). Are risk preferences stable? comparing an experimental measure with a validated survey-based measure. Journal of Risk and Uncertainty, 39, 137–160.CrossRefGoogle Scholar
  6. Bellemare, C., & Kröger, S. (2007). On representative social capital. European Economic Review, 51(1), 183–202.CrossRefGoogle Scholar
  7. Bellemare, C., Kröger, S., van Soest, A. (2008). Measuring inequity aversion in a heterogeneous population using experimental decisions and subjective probabilities. Econometrica, 76(4), 815–839.CrossRefGoogle Scholar
  8. Benartzi, S., & Thaler, R.H. (1995). Myopic loss aversion and the equity premium puzzle. Quarterly Journal of Economics, 110(1), 73–92.CrossRefGoogle Scholar
  9. Binswanger, H.P. (1980). Attitudes towards risk: an experimental measurement in rural India. American Journal of Agricultural Economics, 62, 395–407.CrossRefGoogle Scholar
  10. Blavatskyy, P. (2009). Betting on own knowledge: experimental test of overconfidence. Journal of Risk and Uncertainty, 38, 39–49.CrossRefGoogle Scholar
  11. Bleichrodt, H., Pinto, J.L., Wakker, P.P. (2001). Making descriptive use of prospect theory to improve the prescriptive use of expected utility. Management Science, 47(11), 1498–1514.CrossRefGoogle Scholar
  12. Blundell, R.W., & Stoker, T.M. (2007). Models of aggregate economic relationships that account for heterogeneity. In J.J. Heckman, & E.E. Leamer (Eds.), Handbook of econometrics, handbook of econometrics (Vol. 6, Part 1, chap. 68, pp. 4609–4666). Elsevier.Google Scholar
  13. Browning, M., Hansen, L.P., Heckman, J.J. (1999). Micro data and general equilibrium models. In J.B. Taylor, & M. Woodford (Eds.), Handbook of macroeconomics (Vol. 1, chap. 8, pp. 543–633). Elsevier.Google Scholar
  14. bwGRiD (2007–2011). Member of the German D-Grid initiative, funded by the Ministry of Education and Research (Bundesministerium für Bildung und Forschung) and the Ministry for Science, Research and Arts Baden-Württemberg (Ministerium für Wissenschaft, Forschung und Kunst Baden-Württemberg).
  15. Choi, S., Fisman, R., Gale, D., Kariv, S. (2007). Consistency and heterogeneity of individual behavior under uncertainty. American Economic Review, 97(5), 1921–1938.CrossRefGoogle Scholar
  16. Choi, S., Kariv, S., Müller, W., Silverman, D. (2011). Who is (more) rational? NBER Working Paper 16791.Google Scholar
  17. Coble, K., & Lusk, J. (2010). At the nexus of risk and time preferences: an experimental investigation. Journal of Risk and Uncertainty, 41, 67–79.CrossRefGoogle Scholar
  18. Dave, C., Eckel, C., Johnson, C., Rojas, C. (2010). Eliciting risk preferences: when is simple better? Journal of Risk and Uncertainty, 41, 219–243.CrossRefGoogle Scholar
  19. Dohmen, T., Falk, A., Huffman, D., Sunde, U., Schupp, J., Wagner, G.G. (2005). Individual risk attitudes: New evidence from a large, representative, experimentally-validated survey. IZA Discussion Paper No. 1730.Google Scholar
  20. Dohmen, T., Falk, A., Huffman, D., Sunde, U., Schupp, J., Wagner, G.G. (2011). Individual risk attitudes: measurement, determinants and behavioral consequences. Journal of the European Economic Association, 9(3), 522–550.CrossRefGoogle Scholar
  21. Donkers, B., Melenberg, B., van Soest, A. (2001). Estimating risk attitudes using lotteries; a large sample approach. Journal of Risk and Uncertainty, 22(2), 165–195.CrossRefGoogle Scholar
  22. Falk, A., & Heckman, J.J. (2009). Lab experiments are a major source of knowledge in the social sciences. Science, 326(5952), 535–538.CrossRefGoogle Scholar
  23. Fehr, E., Fischbacher, U., von Rosenbladt, B., Schupp, J., Wagner, G.G. (2003). A nation-wide laboratory: Examining trust and trustworthiness by integrating behavioral experiments into representative surveys. IZA Discussion Paper No. 715.Google Scholar
  24. von Gaudecker, H.M., van Soest, A., Wengström, E. (2008). Selection and mode effects in risk preference elicitation experiments. IZA Discussion Paper No. 3321.Google Scholar
  25. von Gaudecker, H.M., van Soest, A., Wengström, E. (2011). Heterogeneity in risky choice behaviour in a broad population. American Economic Review, 101(2), 664–694.CrossRefGoogle Scholar
  26. Güth, W., Schmidt, C., Sutter, M. (2007). Bargaining outside the lab—a newspaper experiment of a three person-ultimatum game. Economic Journal, 117(518), 449–469.CrossRefGoogle Scholar
  27. Harless, D.W., & Camerer, C.F. (1994). The predictive utility of generalized expected utility theories. Econometrica, 62(6), 1251–1289.CrossRefGoogle Scholar
  28. Harrison, G.W., & List, J.A. (2004). Field experiments. Journal of Economic Literature, 42(4), 1009–1055.CrossRefGoogle Scholar
  29. Harrison, G.W., Lau, M.I., Williams, M.B. (2002). Estimating discount rates in Denmark: a field experiment. American Economic Review, 92(5), 1606–1617.CrossRefGoogle Scholar
  30. Harrison, G.W., Lau, M.I., Rutström, E.E. (2009). Risk attitudes, randomization to treatment, and self-selection into experiments. Journal of Economic Behavior & Organization, 70(3), 498–507.CrossRefGoogle Scholar
  31. Heckman, J.J. (1974). Shadow prices, market wages, and labor supply. Econometrica, 42(4), 679–694.CrossRefGoogle Scholar
  32. Heckman, J.J. (1976). The common structure of statistical models of truncation, sample selection and limited dependent variables and a simple estimator for such models. Annals of Economic and Social Measurement, 5(4), 475–492.Google Scholar
  33. Heckman, J.J. (1979). Sample selection bias as a specification error. Econometrica, 47(1), 153–161.CrossRefGoogle Scholar
  34. Hey, J.D., & Orme, C. (1994). Investigating generalizations of expected utility theory using experimental data. Econometrica, 62(6), 1291–1326.CrossRefGoogle Scholar
  35. Holt, C.A., & Laury, S.K. (2002). Risk aversion and incentive effects. American Economic Review, 92, 1644–1655.CrossRefGoogle Scholar
  36. Hoogendoorn, A.W., & Daalmans, J. (2009). Nonresponse in the recruitment of an internet panel based on probability sampling. Survey Research Methods, 3(2), 59–72.Google Scholar
  37. Huck, S., & Müller, W. (2012). Allais for all: revisiting the paradox in a large representative sample. Journal of Risk and Uncertainty, 44, 261–293.CrossRefGoogle Scholar
  38. Jacobson, S., & Petrie, R. (2009). Learning from mistakes: what do inconsistent choices over risk tell us? Journal of Risk and Uncertainty, 38, 143–158.CrossRefGoogle Scholar
  39. Kahneman, D., & Tversky, A.V. (1979). Prospect theory: an analysis of decision under risk. Econometrica, 47, 263–291.CrossRefGoogle Scholar
  40. Köbberling, V., & Wakker, P.P. (2005). An index of loss aversion. Journal of Economic Theory, 122, 119–131.CrossRefGoogle Scholar
  41. Kreps, D.M., & Porteus, E.L. (1978). Temporal resolution of uncertainty and dynamic choice theory. Econometrica, 46, 185–200.CrossRefGoogle Scholar
  42. Lazear, E., Malmendier, U., Weber, R. (2011). Sorting, prices, and social preferences. American Economic Journal: Applied Economics, 4(1), 136–63.CrossRefGoogle Scholar
  43. Levitt, S.D., & List, J.A. (2007). What do laboratory experiments measuring social preferences reveal about the real world? Journal of Economic Perspectives, 21(2), 153–174.CrossRefGoogle Scholar
  44. Little, R.J., & Rubin, D.B. (2002). Statistical analysis with missing data (2nd edn). New York: John Wiley & Sons Inc.Google Scholar
  45. Loomes, G. (2005). Modelling the stochastic component of behaviour in experiments: some issues for the interpretation of data. Experimental Economics, 8(4), 301–323.CrossRefGoogle Scholar
  46. Loomes, G., Moffatt, P.G., Sugden, R. (2002). A microeconometric test of alternative stochastic theories of risky choice. Journal of Risk and Uncertainty, 24(2), 103–130.CrossRefGoogle Scholar
  47. Lucking-Reiley, D. (1999). Using field experiments to test equivalence between auction formats: magic on the internet. American Economic Review, 89(5), 1063–1080.CrossRefGoogle Scholar
  48. Madrian, B.C., & Shea, D.F. (2001). The power of suggestion: Inertia in 401(k) participation and savings behavior. Quarterly Journal of Economics, 116(4), 1149–1187.CrossRefGoogle Scholar
  49. Revelt, D., & Train, K.E. (2000). Customer-specific taste parameters and mixed logit: Households’ choice of electricity supplier. University of California at Berkeley, Economics Working Paper E00-274.Google Scholar
  50. de Roos, N., & Sarafidis, Y. (2010). Decision making under risk in deal or no deal. Journal of Applied Econometrics, 25(6), 987–1027.CrossRefGoogle Scholar
  51. Starmer, C. (2000). Developments in non-expected utility theory: the hunt for a descriptive theory of choice under risk. Journal of Economic Literature, 38(2), 332–382.CrossRefGoogle Scholar
  52. Tanaka, T., Camerer, C.F., Nguyen, Q. (2010). Risk and time preferences: linking experimental and household survey data from Vietnam. American Economic Review, 100(1), 557–571.CrossRefGoogle Scholar
  53. Tversky, A.V., & Kahneman, D. (1992). Advances in prospect theory: cumulative representation of uncertainty. Journal of Risk and Uncertainty, 5(4), 297–323.CrossRefGoogle Scholar
  54. Viscusi, W.K., Huber, J., Bell, J. (2008). The economic value of water quality. Environmental & Resource Economics, 41(2), 169–187.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2012

Authors and Affiliations

  • Hans-Martin von Gaudecker
    • 1
  • Arthur van Soest
    • 2
  • Erik Wengström
    • 3
    • 4
    Email author
  1. 1.Department of EconomicsUniversität BonnBonnGermany
  2. 2.Department of Econometrics and Operations Research, and NetsparTilburg UniversityTilburgThe Netherlands
  3. 3.Department of EconomicsLund UniversityLundSweden
  4. 4.Department of EconomicsUniversity of CopenhagenCopenhagenDenmark

Personalised recommendations