Abstract
This paper analyzes within-session test/retest data from four different tasks used to elicit risk attitudes. Maximum-likelihood and non-parametric estimations on 16 datasets reveal that, irrespective of the task, measurement error accounts for approximately 50% of the variance of the observed variable capturing risk attitudes. The consequences of this large noise element are evaluated by means of simulations. First, as predicted by theory, the coefficient on the risk measure in univariate OLS regressions is attenuated to approximately half of its true value, irrespective of the sample size. Second, the risk-attitude measure may spuriously appear to be insignificant, especially in small samples. Unlike the measurement error arising from within-individual variability, rounding has little influence on significance and biases. In the last part, we show that instrumental-variable estimation and the ORIV method, developed by Gillen et al. (2019), both of which require test/retest data, can eliminate the attenuation bias, but do not fully solve the insignificance problem in small samples. Increasing the number of observations to N=500 removes most of the insignificance issues.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
As a robustness check we also simulate a probit model.
Jacobson and Petrie (2009) record a large number of such mistakes in a different experiment, and argue that they can provide information about the true population distribution of the risk-aversion coefficient.
The debates around this standard decision model are beyond the scope of the current paper; see O’Donoghue and Somerville (2018) for a recent discussion.
The same method was used by Beauchamp et al. (2017) in a related analysis.
\(\sigma _{x}^{2}\) is the variance of \(x^{*}\) after truncation.
In the Online Appendix C we discuss the plausibility of the normality assumption.
In extreme cases we can observe 0 (resp \(M_t\)) if \(x^{*}+\epsilon <0\) (resp \(x^{*}+\epsilon \le \lfloor {M_t}\rfloor +1\)). We take this into account in the ML estimation.
Precisely, for \(\beta =\{0.15,0.20,0.25,0.30,0.35\}\), we get Corr\((y^{*}\),\(x^{*})=\{0.157,0.207,0.256,0.303,0.347\}\)
References
Andersen, S., Harrison, G. W., Lau, M. I., & Elisabet Rutström, E. (2008). Lost in state space: Are preferences stable? International Economic Review, 49(3), 1091–1112.
Andreoni, J., & Harbaugh, W.T. (2010). Unexpected utility: Experimental tests of five key questions about preferences over risk. University of Oregon Economics Department Working Papers 2010-14, University of Oregon Economics Department. https://ideas.repec.org/p/ore/uoecwp/2010-14.html
Attanasi, G., Georgantzís, N., Rotondi, V., & Vigani, D. (2018). Lottery-and survey-based risk attitudes linked through a multichoice elicitation task. Theory and Decision, 84(3), 341–372.
Bardsley, N., Cubitt, R., Loomes, G., Moffat, P., Starmer, C., & Sugden, R. (2010). Experimental economics: Rethinking the rules. Princeton University Press.
Beauchamp, J. P., Cesarini, D., & Johannesson, M. (2017). The psychometric and empirical properties of measures of risk preferences. Journal of Risk and Uncertainty, 54(3), 203–237.
Blavatskyy, P. R., & Maafi, H. (2018). Estimating representations of time preferences and models of probabilistic intertemporal choice on experimental data. Journal of Risk and Uncertainty, 56(3), 259–287.
Charness, G., Garcia, T., Offerman, T., & Villeval, M. C. (2020). Do measures of risk attitude in the laboratory predict behavior under risk in and outside of the laboratory? Journal of Risk and Uncertainty, 60(2), 99–123.
Chuang, Y., & Schechter, L. (2015). Stability of experimental and survey measures of risk, time, and social preferences: A review and some new results. Journal of Development Economics, 117, 151–170.
Crosetto, P., & Filippin, A. (2013). The “bomb” risk elicitation task. Journal of Risk and Uncertainty, 47(1), 31–65.
Crosetto, P., & Filippin, A. (2016). A theoretical and experimental appraisal of four risk elicitation methods. Experimental Economics, 19(3), 613–641.
Csermely, T., & Rabas, A. (2016). How to reveal people’s preferences: Comparing time consistency and predictive power of multiple price list risk elicitation methods. Journal of Risk and Uncertainty, 53(2–3), 107–136.
Deck, C., Lee, J., Reyes, J., & Rosen, C. (2013). A failed attempt to explain within subject variation in risk taking behavior using domain specific risk attitudes. Journal of Economic Behavior & Organization 87, 1–24, https://doi.org/10.1016/j.jebo.2012.11.010
Dulleck, U., Fooken, J., & Fell, J. (2015). Within-subject intra-and inter-method consistency of two experimental risk attitude elicitation methods. German Economic Review, 16(1), 104–121.
Engel, C., & Kirchkamp, O. (2019). How to deal with inconsistent choices on multiple price lists. Journal of Economic Behavior & Organization, 160, 138–157.
García-Gallego, A., Georgantzís, N., Navarro-Martínez, D., & Sabater-Grande, G. (2011). The stochastic component in choice and regression to the mean. Theory and Decision, 71(2), 251–267.
Gillen, B., Snowberg, E., & Yariv, L. (2019). Experimenting with measurement error: Techniques with applications to the Caltech cohort study. Journal of Political Economy, 127(4), 1826–1863.
Hey, J. D., Morone, A., & Schmidt, U. (2009). Noise and bias in eliciting preferences. Journal of Risk and Uncertainty, 39(3), 213–235.
Holt, C. A., & Laury, S. K. (2002). Risk aversion and incentive effects. American Economic Review, 92(5), 1644–1655.
Jacobson, S., & Petrie, R. (2009). Learning from mistakes: What do inconsistent choices over risk tell us? Journal of Risk and Uncertainty, 38(2), 143–158.
Lönnqvist, J. E., Verkasalo, M., Walkowitz, G., & Wichardt, P. C. (2015). Measuring individual risk attitudes in the lab: Task or ask? An empirical comparison. Journal of Economic Behavior and Organization, 119, 254–266.
Mata, R., Frey, R., Richter, D., Schupp, J., & Hertwig, R. (2018). Risk preference: A view from psychology. Journal of Economic Perspectives, 32(2), 155–72.
Meier, S., & Sprenger, C. D. (2015). Temporal stability of time preferences. Review of Economics and Statistics, 97(2), 273–286.
O’Donoghue, T., & Somerville, J. (2018). Modeling risk aversion in economics. Journal of Economic Perspectives, 32(2), 91–114.
Pedroni, A., Frey, R., Bruhin, A., Dutilh, G., Hertwig, R., & Rieskamp, J. (2017). The risk elicitation puzzle. Nature Human Behaviour, 1(11), 803–809.
Pischke, S. (2007). Lecture notes on measurement error. Mimeo, London School of Economics, London.
Sabater-Grande, G., & Georgantzis, N. (2002). Accounting for risk aversion in repeated prisoners’ dilemma games: An experimental test. Journal of Economic Behavior & Organization, 48(1), 37–50.
Schildberg-Hörisch, H. (2018). Are risk preferences stable? Journal of Economic Perspectives, 32(2), 135–54.
Slovic, P. (1964). Assessment of risk taking behavior. Psychological Bulletin, 61(3), 220.
Wölbert, E., & Riedl, A. (2013). Measuring time and risk preferences: Reliability, stability, domain specificity. Maastricht University, Graduate School of Business and Economics. GSBE Research Memoranda No. 041. https://doi.org/10.26481/umagsb.2013041
Zhou, W., & Hey, J. (2018). Context matters. Experimental Economics, 21(4), 723–756.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
The authors are grateful to an anonymous referee, Olivier Armantier, Gwen-Jiro Clochard, Paolo Crosetto, Tamas Csermely, Jules Depersin, Delphine Dubart, Uwe Dulleck, Antonio Filippin, Jonas Fooken, Nikolaos Georgantzís, Lucas Girard, Yannick Guyonvarch, Xavier d’Haultfoeuille, Nicolas Jacquemet, Alexander Rabas, Gerardo Sabater-Grande, Tara White and participants at the 10th International Conference of the ASFEE 2019 in Toulouse and the ESA European meeting 2019 in Dijon for their suggestions and remarks that have helped to improve this work.
Supplementary Information
Below is the link to the electronic supplementary material.
Appendix A. Simulations for large samples
Appendix A. Simulations for large samples
In this Appendix we provide estimates for large samples: N=300, N=500 and N=1000 (over 10000 simulations) (See Tables 9, 10 and 11).
Rights and permissions
About this article
Cite this article
Perez, F., Hollard, G. & Vranceanu, R. How serious is the measurement-error problem in risk-aversion tasks?. J Risk Uncertain 63, 319–342 (2021). https://doi.org/10.1007/s11166-021-09366-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11166-021-09366-5