Skip to main content
SpringerLink
Log in

A theoretical and experimental appraisal of four risk elicitation methods

  • Original Paper
  • Published:
Experimental Economics Aims and scope Submit manuscript

Abstract

The paper performs an in-depth comparison of four incentivised risk elicitation tasks. We show by means of a simulation exercise that part of the often observed heterogeneity of estimates across tasks is due to task-specific measurement error induced by the mere mechanics of the tasks. We run a replication experiment in a homogeneous subject pool using a between subjects one-shot design. Results shows that the task estimates vary over and above what can be explained by the simulations. We investigate the possibility the tasks elicit different types of preferences, rather than simply provide a different measure of the same preferences. In particular, the availability of a riskless alternative plays a prominent role helping to explain part of the differences in the estimated preferences.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. For an extensive review, including other elicitation tasks with respect to those analysed here (e.g., random lottery pairs as in Hey and Orme (1994), the Becker–DeGroot–Marschak mechanism, auctions, and the trade-off method as in Wakker and Deneffe (1996), see Harrison and Rutström (2008), who underline pros and cons and provide different estimation techniques for the risk preference parameter(s) of different theories.

  2. While an absolute measure of the cognitive load is difficult to establish, it can be reasonably argued that task involving one or a few choices among clearly spelled out alternatives are less demanding than procedures implying dozens of choices, or mechanisms like the BDM requiring to expose the subjects to complicated instructions.

  3. Using the data of the repeated treatment of Crosetto and Filippin (2013), in which the same task, the BRET, was repeated five times we find that the average choice is not significantly different than that of other subjects who played the same task in the one-shot mode. However, the correlation across periods turns out to be, on average, \(\rho \cong 0.35\), ranging from \(\rho \cong 0.01\) to \(\rho \cong 0.6\), i.e., not much higher than what other contributions in the literature found using different elicitation methods. Hence, instability of results does not necessarily rely upon the use of different tasks.

  4. To the best of our knowledge the only papers in the literature using a between-subjects design are Charness and Viceisza (2011) and Harrison (1990). The former elicits risk attitudes of 91 farmers in rural Senegal using HL and GP besides asking the SOEP question, but the fraction of inconsistent choices above \(70\,\%\) in the HL task makes the data not comparable. The latter finds that the BDM displays stronger risk seeking preferences as compared to risk aversion coefficients implicit in first price auction bids.

  5. The interested reader can find in Csermely and Rabas (2014) a comparison of different versions of a Multiple Price List that mimic several elicitation methods, including those analyzed here. The manipulations in Csermely and Rabas (2014) investigate the role played by some features of the payoffs and by fixed versus changing probabilities. In contrast, other characteristics such as the number of choices and how they map into coefficients of relative risk aversion are kept constant across MPL and therefore changed as compared to the original version of each task, with the exception of HL that is used as a benchmark. Their results differ considerably with respect to what usually found in the literature: for instance choices are on average significantly more risk averse in HL than in EG. Such a discrepancy indirectly shows that the framing of the elicitation methods play a role at least as important as that of the fundamentals defined by the underlying lotteries.

  6. The values are based on the baseline of Holt and Laury (2002), doubled to make them comparable with the other tasks.

  7. For the correct formulation of the utility function when r is non-positive see Wakker (2008).

  8. The version of the EG task implemented in Dave et al. (2010) features an additional lottery characterised by the same expected value as the fifth lottery, but by a higher variance. The additional choice reduces the problem because it allows to separate the behaviour of slightly risk-averse agents from that of risk seekers, but it does not solve it since a risk-neutral agent would still be indifferent between the two.

  9. Both Andersen et al. (2006) and Filippin and Crosetto (2015) show that the menu of lotteries available affects choices in the HL task. Crosetto and Filippin (2014) find that removing the first lottery in the EG results in the whole distribution of choices shifting towards more risk-loving decisions. There is similar evidence that subjects tend to avoid boundary choices in other branches of the literature, too (List 2007; Bardsley 2008).

  10. Making zero risky choices is a dominated action and is inconsistent with any degree of risk aversion, given that it implies that the subject prefers 4 euro for sure to 7.7 euro for sure. Note that usually the HL task is summarised by the number of safe choices. However, we prefer to use the number of risky choices for the sake of consistency with the other tasks so that in all the elicitation methods a higher choice represents lower risk aversion.

  11. Only one value of r can be meaningfully computed for the intervals at the extremes, the other being \(r \rightarrow \pm \infty \). We assigned in these cases the only computable boundary of r for choices implying implausible degrees of risk aversion, and a value equal to one, i.e., risk neutrality, to the upper interval of EG and GP where different types of preferences converge.

  12. Note however that adding a stochastic error is not equivalent to having a flatter normal that generates deterministic choices, because only in the first case we can test how each task reacts to preferences that are not perfectly defined.

  13. Our experimental data show that indeed no real subject in GP and BRET submitted such extreme choices. On the other hand, a much larger share of experimental subject choose the safe lottery in EG.

  14. The English translation of the original German instructions is attached in Online supplementary material.

  15. Data from inconsistent subjects are sometimes used in the analyses adopting different techniques. The simplest approach is counting the number of safe choices irrespective of the inconsistency, and is not methodologically sound in our opinion. When building structural models that include a tremble, or when imposing only lower and upper bounds to the risk parameter, data of multiple switchers can instead be meaningfully exploited.

  16. Reporting the median choice in tasks with a low number of categories could result in the need to interpolate the data within the interval of risk aversion in which the median choice falls. In both HL and EG, though, the median choice falls by chance very close to a cutoff point and therefore no interpolation is necessary.

  17. This set of parameters leads to a \(\hat{r}\) distribution which is similar to what emerges by finding the best-fitting parameters from our experimental data N(0.7123, 0.802).

  18. The Do-Investment and the Do-Gamble are added together because belonging to the same domain.

  19. No task correlates significantly with the Dospert subscales for health-related, ethical and recreational risk, and hence we do not include them in Table 7.

  20. An exception is the aforementioned contribution of Dohmen et al. (2011), who compare the SOEP question with an incentivised lottery scheme. Also in their case, however, the fraction of variance explained is fairly low (about \(6\,\%\)).

  21. Ideally, the tasks should capture relevant features of real-life risky behaviour, something that we do not investigate in this paper, but that is definitely an interesting line of future reseach.

References

  • Abdellaoui, M., Driouchi, A., & L’Haridon, O. (2011). Risk aversion elicitation: Reconciling tractability and bias minimization. Theory and Decision, 71, 63–80.

    Article  Google Scholar 

  • Andersen, S., Harrison, G., Lau, M., & Rutström, E. (2006). Elicitation using multiple price list formats. Experimental Economics, 9, 383–405. doi:10.1007/s10683-006-7055-6.

    Article  Google Scholar 

  • Anderson, L. R., & Mellor, J. M. (2009). Are risk preferences stable? Comparing an experimental measure with a validated survey-based measure. Journal of Risk and Uncertainty, 39(2), 137–160.

    Article  Google Scholar 

  • Andreoni, J., & Sprenger, C. (2011). Uncertainty equivalents: Testing the limits of the independence axiom. NBER Working Papers 17342, National Bureau of Economic Research, Inc.

  • Andreoni, J., & Sprenger, C. (2012). Risk preferences are not time preferences. American Economic Review, 102(7), 3357–76.

    Article  Google Scholar 

  • Ball, S., Eckel, C., & Heracleous, M. (2010). Risk aversion and physical prowess: Prediction, choice and bias. Journal of Risk and Uncertainty, 41(3), 167–193.

    Article  Google Scholar 

  • Bardsley, N. (2008). Dictator game giving: Altruism or artefact? Experimental Economics, 11(2), 122–133.

    Article  Google Scholar 

  • Barsky, R. B., Juster, F. T., Kimball, M. S., & Shapiro, M. D. (1997). Preference parameters and behavioral heterogeneity: An experimental approach in the health and retirement study. The Quarterly Journal of Economics, 112(2), 537–579.

    Article  Google Scholar 

  • Becker, G., DeGroot, M., & Marschak, J. (1964). Measuring utility by a single-response sequential method. Behavioral Science, 9, 226–236.

    Article  Google Scholar 

  • Binswanger, H. P. (1981). Attitudes toward risk: Theoretical implications of an experiment in rural India. The Economic Journal, 91(364), 867–890.

    Article  Google Scholar 

  • Blais, A. R., & Weber, E. U. (2006). A domain-specific risk-taking (DOSPERT) scale for adult populations. Judgment and Decision Making, 1, 33–47.

    Google Scholar 

  • Bruner, D. (2009). Changing the probability versus changing the reward. Experimental Economics, 12(4), 367–385.

    Article  Google Scholar 

  • Camerer, C. F. (1992). Recent tests of generalizations of expected utility theory. In W. Edwards (Ed.), Utility theories: Measurements and applications studies in risk and uncertainty (pp. 207–251). Boston, MA: Kluwer Academic Publishers.

    Chapter  Google Scholar 

  • Charness, G., & Gneezy, U. (2010). Portfolio choice and risk attitudes: An experiment. Economic Inquiry, 48(1), 133–146.

    Article  Google Scholar 

  • Charness, G., & Gneezy, U. (2012). Strong evidence for gender differences in risk taking. Journal of Economic Behavior & Organization, 83(1), 50–58.

    Article  Google Scholar 

  • Charness, G., & Viceisza, A. (2011). Comprehension and risk elicitation in the field: Evidence from rural Senegal. IFPRI discussion papers 1135, International Food Policy Research Institute (IFPRI)

  • Charness, G., Gneezy, U., & Imas, A. (2013). Experiential methods: Eliciting risk preferences. Journal of Economic Behavior & Organization, 87, 43–51.

    Article  Google Scholar 

  • Cox, J. C., Roberson, B., & Smith, V. L. (1982). Theory and behavior of single object auctions. Greenwich: JAI Press.

    Google Scholar 

  • Crosetto, P., & Filippin, A. (2013). The ’bomb’ risk elicitation task. Journal of Risk and Uncertainty, 47(1), 31–65.

    Article  Google Scholar 

  • Crosetto, P., & Filippin, A. (2014). Experimental evidence on the cause of gender differences in risk attitudes. mimeo

  • Croson, R., & Gneezy, U. (2009). Gender differences in preferences. Journal of Economic Literature, 47(2), 448–474.

    Article  Google Scholar 

  • Csermely, T., & Rabas, A. (2014). How to reveal people’s preferences: Comparing time consistency and predictive power of multiple price list risk elicitation methods. Department of economics working papers, Vienna University of Economics and Business, Department of Economics.

  • Dave, C., Eckel, C., Johnson, C., & Rojas, C. (2010). Eliciting risk preferences: When is simple better? Journal of Risk and Uncertainty, 41(3), 219–243.

    Article  Google Scholar 

  • Deck, C., Lee, J., & Reyes, J. (2010) Personality and the consistency of risk taking behavior: Experimental evidence. Working Papers 10–17, Chapman University, Economic Science Institute.

  • Deck, C., Lee, J., Reyes, J. A., & Rosen, C. C. (2013). A failed attempt to explain within subject variation in risk taking behavior using domain specific risk attitudes. Journal of Economic Behavior & Organization, 87, 1–24.

    Article  Google Scholar 

  • Dohmen, T., Falk, A., Huffman, D., Sunde, U., Schupp, J., & Wagner, G. G. (2011). Individual risk attitudes: Measurement, determinants, and behavioral consequences. Journal of the European Economic Association, 9(3), 522–550.

    Article  Google Scholar 

  • Eckel, C. C., & Grossman, P. J. (2002). Sex differences and statistical stereotyping in attitudes toward financial risk. Evolution and Human Behavior, 23(4), 281–295.

    Article  Google Scholar 

  • Eckel, C. C., & Grossman, P. J. (2008a). Forecasting risk attitudes: An experimental study using actual and forecast gamble choices. Journal of Economic Behavior & Organization, 68(1), 1–17.

    Article  Google Scholar 

  • Eckel, C. C., & Grossman, P. J. (2008b). Men, women and risk aversion: Experimental evidence (Vol. 1, pp. 1061–1073, chap 113). Handbook of experimental economics results. Amsterdam: Elsevier.

  • Filippin, A., & Crosetto, P. (2015). A reconsideration of gender differences in risk attitudes. Management Science, forthcoming.

  • Garcia-Gallego, A., Georgantzis, N., Jaramillo-Gutiérrez, A., & Parravano, M. (2012). The lottery-panel task for bi-dimensional parameter-free elicitation of risk attitudes. Technical report, Department of Economic Theory and Economic History of the University of Granada.

  • Gneezy, U., & Potters, J. (1997). An experiment on risk taking and evaluation periods. The Quarterly Journal of Economics, 112(2), 631–645.

    Article  Google Scholar 

  • Greiner, B. (2004). The online recruitment system ORSEE 2.0—A guide for the organization of experiments in economics. Working Paper Series in Economics 10, University of Cologne, Department of Economics.

  • Grossman, P. J., & Lugovskyy, O. (2011). An experimental test of the persistence of gender-based stereotypes. Economic Inquiry, 49(2), 598–611.

  • Harbaugh, W., Krause, K., & Vesterlund, L. (2010). The fourfold pattern of risk attitudes in choice and pricing tasks. The Economic Journal, 120(545), 595–611.

    Article  Google Scholar 

  • Harless, D. W., & Camerer, C. F. (1994). The predictive utility of generalized expected utility theories. Econometrica, 62(6), 1251–1289.

    Article  Google Scholar 

  • Harrison, G., & Swarthout, J. (2014). Experimental payment protocols and the bipolar behaviorist. Theory and Decision, 77(3), 423–438.

    Article  Google Scholar 

  • Harrison, G. W. (1990). Risk attitudes in first-price auction experiments: A Bayesian analysis. The Review of Economics and Statistics, 72(3), 541–546.

    Article  Google Scholar 

  • Harrison, G. W., & Rutström, E. E. (2008). Risk aversion in the laboratory. In J. C. Cox & G. W. Harrison (Eds.), Risk aversion in experiments, research in experimental economics (Vol. 12, pp. 41–196). Bradford: Emerald Group Publishing Limited.

    Chapter  Google Scholar 

  • Harrison, G. W., Lau, M. I., & Rutström, E. E. (2007). Estimating risk attitudes in Denmark: A field experiment. Scandinavian Journal of Economics, 109(2), 341–368.

    Article  Google Scholar 

  • Hey, J. D., & Orme, C. (1994). Investigating generalizations of expected utility theory using experimental data. Econometrica, 62(6), 1291–1326.

    Article  Google Scholar 

  • Holt, C., & Laury, S. (2002). Risk aversion and incentive effects. American Economic Review, 92(5), 1644–1655.

    Article  Google Scholar 

  • Isaac, R., & James, D. (2000). Just who are you calling risk averse? Journal of Risk and Uncertainty, 20(2), 177–187.

    Article  Google Scholar 

  • Kaivanto, K., & Kroll, E. B. (2011). Negative recency, randomization device choice, and reduction of compound lotteries. Working Paper Series in Economics 22, Karlsruhe Institute of Technology (KIT), Department of Economics and Business Engineering.

  • Kaivanto, K., & Kroll, E. B. (2012). Alternation bias and reduction in st. petersburg gambles: An experimental investigation. Technical report, Lancaster University.

  • Lejuez, C., Read, J., Kahler, C., Richards, J., Ramsey, S., Stuart, G., et al. (2002). Evaluation of a behavioral measure of risk taking: The Balloon Analogue Risk Task (BART). Journal of Experimental Psychology: Applied, 8(2), 75.

    Google Scholar 

  • List, J. A. (2007). On the interpretation of giving in dictator games. Journal of Political Economy, 115, 482–493.

    Article  Google Scholar 

  • Loomes, G., & Sugden, R. (1982). Regret theory: An alternative theory of rational choice under uncertainty. Economic Journal, 92(368), 805–824.

    Article  Google Scholar 

  • Nelson, J. A. (2014). Are women really more risk-averse than men? A re-analysis of the literature using expanded methods. Journal of Economic Surveys, 29(3), 566–585.

  • Nelson, J. A. (2015). Not-so-strong evidence for gender differences in risk taking. Feminist Economics, forthcoming.

  • Perugini, M., Tan, J. H. W., & Zizzo, D. J. (2010). Which is the more predictable gender? Public good contribution and personality. Economic Issues Journal Articles, 15(1), 83–110.

    Google Scholar 

  • Reynaud, A., & Couture, S. (2012). Stability of risk preference measures: Results from a field experiment on French farmers. Theory and Decision, 73(2), 203–221.

    Article  Google Scholar 

  • Slovic, P. (1966). Risk-taking in children: Age and sex differences. Child Development, 37(1), 169–176.

    Article  Google Scholar 

  • Starmer, C. (2000). Developments in non-expected utility theory: The hunt for a descriptive theory of choice under risk. Journal of Economic Literature, 38(2), 332–382.

    Article  Google Scholar 

  • van Rossum, G. (1995). Python reference manual. CWI Report CS-R9525.

  • Wagner, G. G., Frick, J.R., & Schupp, J. (2007) The german socio-economic panel study (soep): Scope, evolution and enhancements. SOEPpapers on Multidisciplinary Panel Data Research 1, DIW Berlin, The German Socio-Economic Panel (SOEP).

  • Wakker, P. P., (2008) Explaining the characteristics of the power (CRRA) utility family. Health Economics, 17(12), 1329–1344.

  • Wakker, P., & Deneffe, D. (1996). Eliciting von Neumann-Morgenstern utilities when probabilities are distorted or unknown. Management Science, 42(8), 1131–1150.

    Article  Google Scholar 

Download references

Acknowledgments

We are grateful to the Max Planck Institute of Economics (Jena) for financial and logistic support and to Denise Hornberger, Nadine Marmai, Florian Sturm, and Claudia Zellmann for their assistance in the lab. We would like to thank the members of the ESA mailing list for useful references and participants to seminars in Strasbourg, Middlesex, Paris 1 Sorbonne, MPI Jena, DIW Berlin, INRA Rennes and Göttingen as well as the audience of the IMEBE conference in Madrid and the BEELAB conference in Florence and two anonymous referees for useful comments. All remaining errors are ours.

Author information

Authors and Affiliations

  1. INRA and Univ. Grenoble Alpes, UMR 1215 GAEL, 38000, Grenoble, France

    Paolo Crosetto

  2. DEMM, University of Milan, Via Conservatorio 7, 20122, Milano, Italy

    Antonio Filippin

  3. Institute for the Study of Labor (IZA), Schaumburg-Lippe-Str. 5-9, 53113, Bonn, Germany

    Antonio Filippin

Authors
  1. Paolo Crosetto
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Antonio Filippin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Paolo Crosetto.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (PDF 27 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Crosetto, P., Filippin, A. A theoretical and experimental appraisal of four risk elicitation methods. Exp Econ 19, 613–641 (2016). https://doi.org/10.1007/s10683-015-9457-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10683-015-9457-9

Keywords

JEL Classification

Search

Navigation