Abstract
We study the estimation of risk preferences with experimental data and focus on the trade-offs when choosing between two different elicitation methods that have different degrees of difficulty for subjects. We analyze how and when a simpler, but coarser, elicitation method may be preferred to the more complex, but finer, one. Results indicate that the more complex measure has overall superior predictive accuracy, but its downside is that subjects exhibit noisier behavior. Our main result is that subjects’ numerical skills can help better assess this tradeoff: the simpler task may be preferred for subjects who exhibit low numeracy, as it generates less noisy behavior but similar predictive accuracy. For subjects with higher numerical skills, the greater predictive accuracy of the more complex task more than outweighs the larger noise. We also explore preference heterogeneity and provide methodological suggestions for future work.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
We use the terms “elicitation methods” and “experimental measures” interchangeably throughout the text.
When subjects fail to understand a task in the field, they tend to resort to alternating between options (for example, alternating between the risky and safe gambles in the HL task), and describe their choices with phrases such as “I just wanted to try it.”
The existence of a well-behaved utility function (with precise decision making skills on the part of subjects) has been criticized, for example, by researchers that have found that subjects’ preferences are inconsistent across similar tasks (e.g., Slovic 1962; Isaac and James 2000; Berg et al. 2005; Peters et al. 2006).
Starmer (2000) surveys the symbiotic theoretical and empirical developments in analyzing decision making under risk.
There were 8 types of decisions (three used here), and 100 decisions in total. All subjects were paid at the end of the experimental session, after all decisions were completed, including the (unpaid) math literacy task that we describe below.
Our design does not allow us to investigate order effects directly since all subjects completed the EG task prior to the HL task. A potential drawback for our results is that we cannot determine whether the additional noise observed in HL is due in part to this particular ordering. We deem this possibility as somewhat unlikely as it is not clear how a single decision on a prior gamble (recall that it is only one choice that subjects make in EG) can increase noise later in HL (i.e. one can argue that it may decrease it as subjects are “more focused” on analyzing gambles after seeing the EG task).
A person choosing Gamble 3, for example, would have a coefficient of relative risk aversion in the range 0.71–1.16: a person with r = 0.71 would be just indifferent between Gambles 3 and 4, and a person with r = 1.16 is just indifferent between Gambles 2 and 3.
The average raw score for our sample (out of a possible 500) is 281.25 (SEM = 1.7). For the Canadian population the average is 272.3 (SEM = 0.7) (Statistics Canada 2003, p. 49).
Engle-Warnick et al. (2005) conduct experiments where subjects complete both a binary-choice version and a single-choice version of the EG task and report that most subjects make equivalent choices. We also have conducted both with a sample of students, and found about 85% consistent choices across the two presentations. (Data available on request.)
In Section 4.2 we discuss different “cutoff” scores for classifying low math ability subjects.
We also employed expo-power and power utility functions in our estimation to verify the extent to which our results (reported below) were sensitive to the assumed functional form for utility. We ran into several problems in such specifications: a) convergence was difficult to achieve, b) estimates produced fitted values that were outside the range of observed data, and c) estimates were at times inconsistent with economic theory. We nevertheless conducted robustness checks with a Fechner error specification (see Hey and Orme 1994) in lieu of the Luce specification in (1) which included linear, logarithmic, power and expo-power functional forms of utility; our main findings were confirmed in such estimations (results can be made available upon request).
We also implemented interval regressions to estimate risk stances with and without inconsistent subjects; however we do not report such results as this specification does not contemplate the ‘noise’ associated with the data (the focus of our study).
Four additional subjects chose the safe outcome for all ten HL decisions. Excluding them does not change the results.
For medium and high literacy people in HL, the range of predicted noise is closer to that of EG but still with cumulative distributions to the right of that of EG (not shown).
These predictive accuracy values are nearly identical when inconsistent subjects are removed (i.e. using Table 7 estimates).
If, instead, Decisions 1, 2, 3, 4 and 10 are excluded, a similar shape is obtained.
Predictive accuracy is inferior if Decisions 1–4 and 10 are excluded: 0.74 for all individuals and 0.63 for low math ability people. The reason for this is that there is a higher predictive accuracy for Decision 3 than for Decision 9. Thus, excluding Decisions 1, 2, 4, 9 and 10 is a relatively conservative way to collapse HL into 5 decisions.
See also Eckel 1999 who shows that grade point average is strongly related to deviations from risk neutrality, providing early evidence that cognitive skills may affect elicited risk preferences.
References
Andersen, S., Harrison, G., Lau, M., & Rutström, E. (2008). Eliciting risk and time preferences. Econometrica, 76(3), 583–618.
Barr, A., & Genicot, G. (2008). Risk sharing, commitment, and information: An experimental analysis. Journal of the European Economic Association, 6(6), 1151–1185.
Becker, G., DeGroot, M. H., & Marschak, J. (1964). Measuring utility by a single-response sequential method. Behavioral Science, 9, 226–232.
Berg, J., Dickhaut, J., & McCabe, K. (2005). Risk preference instability across institutions: A dilemma. Proceedings of the National Academies of Science, 102, 4209–4214.
Binswanger, H. (1980). Attitudes toward risk: Experimental measurement in rural India. American Journal of Agricultural Economics, 62, 395–407.
Binswanger, H. (1981). Attitudes toward risk: Theoretical implications of an experiment in rural India. Economic Journal, 91, 867–890.
Bohm, P., Linden, J., & Sonnegard, J. (1997). Eliciting reservation prices: Becker-DeGroot-Marschak mechanisms vs. markets. Economic Journal, 107, 1079–1089.
Burks, S., Carpenter, J., Götte, L., & Rustichini, A. (2009). Cognitive skills explain economic preferences, strategic behavior, and job attachment. Proceedings of the National Academy of Sciences, 106(19), 7745–7750.
Dohmen, T., Falk, A., Huffman, D., & Sunde, U. (2010). Are risk aversion and impatience related to cognitive ability? American Economic Review, 100(3), 1238–1260.
Eckel, C. (1999). Commentary on “The effects of financial incentives in experiments: A review and capital-labor production framework.” Journal of Risk and Uncertainty, 19(1–3), 47–48.
Eckel, C. (2007). Experiments on gender differences. In S. N. Durlauf & L. E. Blume (Eds.), The New Palgrave Dictionary of Economics (2nd ed.). Basingstoke: Palgrave Macmillan.
Eckel, C., & Grossman, P. (2002). Sex differences and statistical stereotyping in attitudes toward financial risk. Evolution and Human Behavior, 23(4), 281–295.
Eckel, C., & Grossman, P. (2008). Forecasting risk attitudes: An experimental study using actual and forecast gamble choices. Journal of Economic Behavior and Organization, 68(1), 1–17.
Eckel, C., Grossman, P., Johnson, C., Milano, A., Rojas, C., and Wilson, R. (2007). Explaining risk preferences in high school students: A preliminary look. CBEES Working paper, University of Texas at Dallas.
Eisenberger, R., & Weber, M. (1995). Willingness-to-pay and willingness-to-accept for risky and ambiguous lotteries. Journal of Risk and Uncertainty, 10, 223–33.
Engle-Warnick, J., Escobal, J., and Laszlo, S. (2005). The effect of an additional alternative on measured risk preferences in a field experiment. Unpublished manuscript. Experimental 0511003, EconWPA.
Harbaugh, W., Krause, K., & Vesterlund, Lise. (2002). Risk attitudes of children and adults: Choices over small and large probability gains and losses. Experimental Economics, 5(1), 53–84.
Harbaugh, W. T., Krause, K., & Vesterlund, L. (2010). The fourfold pattern of risk attitudes in choice and pricing tasks. The Economic Journal, 120(545), 595–611.
Harrison, G. (1986). An experimental test for risk aversion. Economics Letters, 21, 7–11.
Harrison, G. (1992). Theory and misbehavior of first-price auctions: Reply. American Economic Review, 82, 1426–1443.
Hey, J., & Orme, C. (1994). Investigating generalizations of expected utility using experimental data. Econometrica, 62(6), 1291–1326.
Holt, C., & Laury, S. (2002). Risk aversion and incentive effects. American Economic Review, 92(5), 1644–1655.
Horowitz, J. (2006). The Becker-DeGroot-Marschak mechanism is not necessarily incentive compatible, even for non-random goods. Economics Letters, 93(1), 6–11.
Huck, S., & Weizsacker, G. (1999). Risk, complexity, and deviations from expected value maximization. Journal of Economic Psychology, 20, 699–715.
Isaac, M., & James, D. (2000). Just who are you calling risk averse? Journal of Risk and Uncertainty, 20(2), 177–187.
Johnson, C., Montmarquette, C., and Eckel, C. (2003). Fostering adult education: A laboratory experiment on the efficient use of loans, grants, and saving incentives. Social Research and Demonstration Corporation.
Johnson, C., Eckel, C., and Engle-Warnick, J. (2007). An adaptive instrument to elicit risk preferences. Working paper, University of Montreal, 2007.
Kachelmeier, S., & Shehata, M. (1992). Examining risk preferences under high monetary incentives: Experimental evidence from the People's Republic of China. American Economic Review, 82, 1120–1141.
Karni, E., & Safra, Z. (1987). Preference reversal and the observability of preferences by experimental methods. Econometrica, 55(3), 675–685.
Peters, E., Vastfjall, D., Slovic, P., Mertz, C., Mazzocco, K., & Dicker, S. (2006). Numeracy and Decision Making. Psychological Science, 17(5), 407–413.
Slovic, P. (1962). Convergent Validation of Risk Taking Measures. Journal of Abnormal and Social Psychology, 65(1), 68–71.
Starmer, C. (2000). Developments in non-expected utility theory: The hunt for a descriptive theory of choice under risk. Journal of Economic Literature, 38, 332–382.
Statistics Canada and OECD. (2003). Learning a Living: First Results of the Adult Literacy and Life Skills Survey. http://www.statcan.ca/english/freepub/89-603-XIE/2005001/pdf.htm
Acknowledgements
We thank Glenn Harrison who generously provided his advice and code; Scott Murray and Fernando Cartwright provided valuable assistance. Comments from participants in workshops and conferences (ASSA, ESA, Dallas Federal Reserve Board, Georgia State University, Texas A&M University, and University of Arizona) substantially improved the paper. The experiments were funded by Canada Student Loans Program Directorate and the Applied Research Branch of Human Resources Development Canada (HRDC). Statistics Canada provided the numeracy assessment. Support for the research was provided by the John D. and Catherine T. MacArthur Foundation, Network on Preferences and Norms. Eckel was supported by the National Science Foundation (SES-0443708, SES-0094800).
Author information
Authors and Affiliations
Corresponding author
Appendix: Decision forms for risk choices
Appendix: Decision forms for risk choices
Eckel-Grossman Risk Task
For Decision 41 you will select from among six different gambles the one gamble you would like to play. The six different gambles are listed below.
-
You must select one and only one of these gambles.
-
To select a gamble place an X in the appropriate box.
Each gamble has two possible outcomes (Roll Low or Roll High) with the indicated probabilities of occurring. Your compensation for this part of the study will be determined by:
-
which of the six gambles you select; and
-
which of the two possible payoffs occur.
For example, if you select Gamble 4 and Roll High occurs, you will be paid $52. If Roll Low occurs, you will paid $16.
For every gamble, each Roll has a 50% chance of occurring.
At the end of the study, if Decision 41 is randomly selected, you will roll a ten-sided die to determine which event will occur. If you roll a 1, 2, 3, 4 or 5, Roll Low will occur. If you roll a 6, 7, 8, 9 or 0, Roll High will occur.
Holt-Laury Risk Task
In this next set of 10 decisions, you are given a chance to earn a cash prize today. For each decision, you will choose between playing two Gambles, A and B. Here is an example:
Each Gamble is composed of two outcomes. Which one occurs depends on the roll of a ten-sided die. For instance, let’s look at Gamble A. You have 3 out of 10 chances to win $40 and 7 out of 10 chances to win $32. If you roll a 1, 2 or 3, (3 chances out of 10) then you win $40. If you roll a 4,5,6,7,8,9,0, (7 chances out of 10) then you win $32.
Rights and permissions
About this article
Cite this article
Dave, C., Eckel, C.C., Johnson, C.A. et al. Eliciting risk preferences: When is simple better?. J Risk Uncertain 41, 219–243 (2010). https://doi.org/10.1007/s11166-010-9103-z
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11166-010-9103-z