Ambiguity preferences are important to explain human decision-making in many areas in economics and finance. To measure individual ambiguity preferences, the experimental economics literature advocates using incentivized laboratory experiments. Yet, laboratory experiments are costly, time-consuming and require substantial administrative effort. This study develops an experimentally validated ambiguity preference survey module that can reliably measure ambiguity preferences when carrying out laboratory experiments is impractical. This toolkit may have wide applications, including end-of-session lab questionnaires, large scale surveys and financial client assessments.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
For example, theoretical models show that ambiguity aversion can explain the lower than expected investment in financial markets (e.g. Dow and Da Costa Werlang 1992; Bossaerts et al. 2010) and real investment projects (e.g. Nishimura and Ozaki 2007), the economic evaluation of climate change (e.g. Weitzman 2009), the equity premium puzzle (e.g. Collard et al. 2018) or low reservation wages for job searchers (e.g. Nishimura and Ozaki 2004.
For example, the study by Dimmock et al. (2016a) paid USD 23,850 in real incentives.
One example is the design by Dimmock et al. (2016b) who use three sets of incentivized dynamic choice tasks to elicit ambiguity preferences for three different levels of ambiguity. They show that a-insensitivity is related to financial behaviour.
In our considerations, there was also a trade-off between the duration of the experiment (and the incentives paid) and the number of participants our budget allowed.
We also considered measuring ambiguity preferences using a binary choice list where the expected payoff of the risky urn changes by monotonically decreasing its prize (but not its probability), see for example Chakravarty and Roy (2009). However, in such a setting the switching point depends on the subjects’ risk preferences. Although it is possible to first estimate risk preferences, such a procedure increases the overall measurement error of the estimated ambiguity preferences.
Any incentivized task can be transformed into a non-incentivized task by removing the monetary incentives. However, the complexity remains higher than in usual thought experiments.
In a recent paper, Bade (2015) presents some problems with this random incentive mechanism when subjects are ambiguity averse. Yet, to our knowledge, the random incentive mechanism remains the best incentive scheme to measure ambiguity preferences and it is commonplace in laboratory and field studies.
The lowest payment was GBP 10, the highest payment GBP 26. Since the sessions lasted for about 40 minutes, the payoffs are sizable.
Because of symmetry of the two colours and an implicitly assumed exchangeability condition (Chew and Sagi 2006), a matching probability of 0.5 is the subjective probability of an ambiguity-neutral decision maker. From this, it follows that a matching probability larger than 0.5 corresponds to ambiguity seeking preferences, and that a matching probability smaller than 0.5 corresponds to ambiguity averse preferences. See Dimmock et al. (2015) for a theoretical derivation.
If a subject is ambiguity neutral (m = 0.5), then he is indifferent between switching after situation 4 or 5, which corresponds to an estimated matching probability of 0.45 or 0.55.
In both thought experiments, the majority of subjects is ambiguity averse (p < 0.01, binomial probability test).
The choice set of alternative item selection procedures is vast. Alternatives vary along the dimensions of i) the selection procedure; ii) the goodness of fit statistics used; iii) the specification used in the iteration. Alternative selection procedures in (i) include stepwise forward selection, stepwise backward selection, or the Lasso technique by Tibshirani (1996). Each of these techniques has considerable drawbacks that make them unsuitable for our purpose. For a critical discussion on these alternative selection procedures see Falk et al. (2016), footnote 13. Alternatives in (ii) include the BIC (which we use), the adjusted R2, AIC or the Mean Squared Error. These statistics apply different penalties for the estimation of additional parameters. Alternatives in (iii) include non-linear functional forms.
In order to classify responses into binary indicators (i.e. ambiguity tolerant/intolerant), we use the sample median value for each survey question to take into account the distribution of sample responses. Some of the survey question responses have skewed distributions. This is likely to be due to the composition of our subject sample: students. For example, the majority of young people agree to the statement “I would like to live in a foreign country new to me.” While the use of the sample median is not indispensable, it creates relative classifications within sample.
Two alternative specifications are possible. A more flexible approach would be to implement a fully non-parametric specification with a binary indicator for each value of the rating scales (= 1 if that value is chosen, = 0 otherwise). The advantage of this approach is to allow for a flexible, non-linear, pattern of predictors’ correlations with the ambiguity benchmark. As we measured the survey item predictors on 7-point Likert scales, this approach would imply estimating 6 parameters for each ordinal predictor. Given our sample size, such an approach would be inappropriate as we would run out of degrees of freedom quickly. However, the fully non-parametric approach would be advisable for future studies with larger samples. A second alternative approach is to treat rating scales as continuous variables. This approach relies on the assumption of linearity, which may, but need not, be satisfied. As a robustness test, we run auxiliary regressions to test the linearity assumption in our sample and we conclude that the linearity assumption cannot be rejected for the items selected. These tests are available in the online Appendix E.
This result shows that the dynamic Ellsberg thought experiment is a better predictor relative to the static two-colour Ellsberg thought experiment. This does not mean that the Ellsberg thought experiment is not a good proxy, but that the dynamic Ellsberg thought experiment explains more variation in the benchmark. Indeed, the static Ellsberg thought experiment is significantly correlated with ambiguity preferences in pairwise tests (see Table 4). This provides a validation to existing applications of the static Ellsberg thought experiment, e.g., Butler et al. (2014).
The experimental procedure of this second round of experiments is identical to those described in Section 2.
As an example, suppose a subject reveals in the dynamic Ellsberg urn experiment a matching probability of 37.5% and answers all 4 survey questions with values revealing ambiguity tolerant attitudes. Then the ambiguity preference score equals 37.5 + 20 + 20 + 20 − 20 + 130 = 207.5.
Abdellaoui, M., Baillon, A., Placido, L., Wakker, P.P. (2011). The rich domain of uncertainty: Source functions and their experimental implementation. American Economic Review, 101(2), 695–723.
Bade, S. (2015). Randomization devices and the elicitation of ambiguity-averse preferences. Journal of Economic Theory, 159, 221–235.
Baillon, A., Wakker, L., Cabantous, P.P. (2012). Aggregating imprecise or conflicting beliefs: an experimental investigation using modern ambiguity theories. Journal of Risk and Uncertainty, 44(2), 115–147.
Baillon, A., & Bleichrodt, H. (2015). Testing ambiguity models through the measurement of probabilities for gains and losses. American Economic Journal: Microeconomics, 7(2), 77–100.
Baillon, A., Huang, Z., Selin, A., Wakker, P.P. (2018). Measuring ambiguty attitudes for all (natural) events. Econometrica, 86(5), 1839–1858.
Becker, G.M., DeGroot, M.H., Marschak, J. (1964). Measuring utility by a single-response sequential method. Behavioral Science, 9(3), 226–232.
Bernard, T., Dercon, S., Orkin, K., Taffesse, A. (2014). The future in mind: Aspirations and forward-looking behaviour in rural Ethiopia. London: Centre for Economic Policy Research.
Bianchi, M, & Tallon, J.-M. (2018). Ambiguity preferences and portfolio choice: Evidence from the field. Management Science.
Binmore, K., Stewart, L., Voorhoeve, A. (2012). How much ambiguity aversion? Finding indifferences between Ellsberg’s risky and ambiguous bets. Journal of Risk and Uncertainty, 45(3), 215–238.
Bossaerts, P., Ghirardato, P., Guarnaschelli, S., Zame, W.R. (2010). Ambiguity in asset markets: Theory and experiment. Review of Financial Studies, 23 (4), 1325–1359.
Budner, S. (1962). Intolerance for ambiguity as a personal variable. Journal of Personality, 30, 29–50.
Butler, J.V., Guiso, L., Jappelli, T. (2014). The role of intuition and reasoning in driving aversion to risk and ambiguity. Theory and Decision, 77, 455–484.
Camerer, C, & Weber, M. (1992). Recent developments in modeling preferences. Journal of Risk and Uncertainty, 5(4), 325–370.
Carbone, E., Dong, X., Hey, J. (2017). Elicitation of preferences under ambiguity. Journal of Risk and Uncertainty, 54(2), 87–102.
Chakravarty, S, & Roy, J. (2009). Recursive expected utility and the separation of attitudes towards risk and ambiguity: An experimental study. Theory and Decision, 66(3), 199–228.
Chang, E.C., Maydeu-Olivares, A., D’Zurilla, T. (1997). Optimism and pessimism as partially independent constructs: Relationship to positive and negative affectivity and psychological well-being. Personality and Individual Differences, 23 (3), 433–440.
Charness, G., Karni, E., Levin, D. (2013). Ambiguity attitudes and social interactions. Journal of Risk and Uncertainty, 46, 1–25.
Chateauneuf, A., Eichberger, J., Grant, S. (2007). Choice under uncertainty with the best and worst in mind: Neo-additive capacities. Journal of Economic Theory, 137, 538–567.
Chew, S.H., & Sagi, J.S. (2006). Event exchangeability: Probabilistic sophistication without continuity or monotonicity. Econometrica, 74(3), 771–786.
Chow, C.C., & Sarin, R.K. (2002). Known, unknown, and unknowable uncertainties. Theory and Decision, 52, 127–138.
Cohen, M, Jaffray, J.-Y., Said, T. (1987). Experimental comparison of individual behavior under risk and under uncertainty for gains and for losses. Organizational Behavior and Human Decision Processes, 39(1), 1–22.
Cohen, M., Tallon, J.-M., Vergnaud, J.-C. (2011). An experimental investigation of imprecision attitude and its relation with risk attitude and impatience. Theory and Decision, 71(1), 81–110.
Cole, S., Giné, X., Tobacman, J., Townsend, R., Topalova, P., Vickery, J. (2013). Barriers to household risk management: Evidence from India. American Economic Journal: Applied Economics, 5(1), 104.
Collard, F., Mukerji, S., Sheppard, K., Tallon, J.-M. (2018). Ambiguity and the historical equity premium. Quantitative Economics, 9, 945–993.
Curley, S.P., & Yates, J.F. (1989). An empirical evaluation of descriptive models of ambiguity reactions in choice situations. Journal of Mathematical Psychology, 33 (4), 397–427.
Dimmock, S.G., Kouwenberg, R., Mitchell, O.S., Peijnenburg, K. (2015). Estimating ambiguity preferences and perceptions in multiple prior models: Evidence from the field. Journal of Risk and Uncertainty, 51(3), 219–244.
Dimmock, S.G., Kouwenberg, R., Mitchell, O.S., Peijnenburg, K. (2016a). Ambiguity aversion and household portfolio choice puzzles: Empirical evidence. Journal of Financial Economics, 119(3), 559–577.
Dimmock, S.G., Kouwenberg, R., Wakker, P.P. (2016b). Ambiguity attitudes in a large representative sample. Management Science, 62(5), 1363–1380.
Dohmen, T., Falk, A., Huffman, D., Sunde, U., Schupp, J., Wagner, G.G. (2011). Individual risk attitudes: Measurement, determinants and behavioral consequences. Journal of the European Economic Association, 9(3), 522–550.
Donkers, B., Melenberg, B., Van Soest, A. (2001). Estimating risk attitudes using lotteries: A large sample approach. Journal of Risk and Uncertainty, 22(2), 165–195.
Dow, J. , & Da Costa Werlang, S.R. (1992). Uncertainty aversion, risk aversion, and the optimal choice of portfolio. Econometrica, 60(1), 197–204.
Ellsberg, D. (1961). Risk, ambiguity and the savage axioms. Quarterly Journal of Economics, 75(4), 643–669.
Falk, A., Becker, A., Dohmen, T., Huffman, D., Sunde, U. (2016). The preference survey module: A validated instrument for measuring risk, time and social preferences. Working paper.
Falk, A., & Zimmermann, F. (2017). Consistency as a signal of skills. Management Science, 63(7), 2197–2210.
Falk, A., Becker, A., Dohmen, T., Enke, B., Huffman, D., Sunde, U. (2018). Global evidence on economic preferences. Quarterly Journal of Economics, 133(4), 1645–1692.
Fehr, E., Fischbacher, U., Rosenbladt, B., Schupp, J., Wagner, G. (2003). A nationwide laboratory: Examining trust and trustworthiness by integrating behavioural experiments into representative surveys. Working paper.
Festinger, L. (1957). A theory of cognitive dissonance. Stanford: Stanford University Press.
Fischbacher, U. (2007). z-tree: Zurich toolbox for ready-made economic experiments. Experimental Economics, 10(2), 171–178.
Fox, C.R., & Tversky, A. (1995). Ambiguity aversion and comparative ignorance. Quarterly Journal of Economics, 110(3), 585–603.
Furnham, A., & Ribchester, T. (1995). Tolerance of ambiguity: A review of the concept, its measurement and applications. Current Psychology, 14(3), 179–199.
Greco, V., & Roger, D. (2001). Coping with uncertainty: The construction and validation of a new measure. Personality and Individual Differences, 31, 519–534.
Guiso, L., & Paiella, M. (2008). Risk aversion, wealth and background risk. Journal of the European Economic Association, 6(6), 1109–1150.
Guiso, L, & Jappelli, T. (2008). Financial literacy and portfolio diversification. Working paper.
Heath, C., & Tversky, A. (1991). Preference and belief: Ambiguity and competence in choice under uncertainty. Journal of Risk and Uncertainty, 4(1), 5–28.
Holt, C.A., & Laury, S.K. (2002). Risk aversion and incentive effects. American Economic Review, 92(5), 1644–1655.
Kirton, M. J. (1981). A reanalysis of two scales of tolerance of ambiguity. Journal of Personality Assessment, 45(4), 407–414.
Knight, F.H. (1921). Risk, uncertainty, and profit. Boston: Houghton Mifflin Company.
Kocher, M.G., Lahno, A.M., Trautmann, S.T. (2015). Ambiguity aversion is the exception. Working paper.
Lauriola, M, & Levin, I.P. (2001). Relating individual differences in attitude toward ambiguity to risky choices. Journal of Behavioral Decision Making, 14(2), 107–122.
MacDonald Jr., A. P. (1970). Revised scale for ambiguity tolerance: Reliability and validity. Psychological Reports, 26, 791–798.
McLain, D.L. (2009). Evidence of the properties of an ambiguity tolerance measure: The multiple stimulus types ambiguity tolerance scale-II (mstat-II). Psychological Reports, 105(3), 975–988.
Nishimura, K.G., & Ozaki, H. (2004). Search and Knightian uncertainty. Journal of Economic Theory, 119(2), 299–333.
Nishimura, K.G., & Ozaki, H. (2007). Irreversible investment and Knightian uncertainty. Journal of Economic Theory, 136(1), 668–694.
Norton, R.W. (1975). Measurement of ambiguity tolerance. Journal of Personality Assessment, 39(6), 607–619.
Robins, R.W., Hendin, H.M., Trzesniewski, K.H. (2001). Measuring global self-esteem: Construct validation of a single-item measure and the Rosenberg self-esteem scale. Personality and Social Bulletin, 27(2), 151–161.
Rydell, S.T., & Rosen, E. (1966). Measurement and some correlates of need cognition. Psychological Reports, 19, 139–165.
Sutter, M., Kocher, M.G., Glätzle-Rüetzler, D., Trautmann, S.T. (2013). Impatience and uncertainty: Experimental decisions predict adolescents’ field behavior. The American Economic Review, 103(1), 510–531.
Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, Series B (Methodological), 58(1), 267–288.
Trautmann, S.T., & van de Kuilen, G. (2016). Ambiguity Attitudes. Blackwell Handbook of Judgment and Decision Making. Malden: Blackwell Publishing.
Vieider, F.M., Lefebvre, M., Bouchouicha, R., Chmura, T., Hakimov, R., Krawczyk, M., Martinsson, P. (2015). Common components of risk and uncertainty attitudes across contexts and domains: Evidence from 30 countries. Journal of the European Economic Association, 13(3), 421–452.
Weitzman, M.L. (2009). On modeling and interpreting the economics of catastrophic climate change. Review of Economics and Statistics, 91(1), 1–19.
We thank the editor Kip Viscusi and an anonymous referee for their constructive comments. We also thank Johannes Abeler, Aurelien Baillon, Han Bleichrodt, Syngjoo Choi, Peter Dürsch, Roy Kouwenberg, Daniel Martin, Ron Smith, Uwe Sunde, Matthias Sutter, Jean-Marc Tallon, Stefan Trautmann, Peter Wakker, Matthias Wibral, seminar participants at Paris-1, as well as conference participants at IMEBESS, the Maastricht behavioral and experimental economics symposium, FUR XVI, Verein für Socialpolitik, ESRC Workshop on Preferences and Personality, NIBS, D-TEA, and the Royal Economic Society conference for helpful discussion and comments. This work was supported by a BEI research grant by Birkbeck College. Any remaining errors are ours.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
About this article
Cite this article
Cavatorta, E., Schröder, D. Measuring ambiguity preferences: A new ambiguity preference survey module. J Risk Uncertain 58, 71–100 (2019). https://doi.org/10.1007/s11166-019-09299-0
- Preference measurement
- Decision making
- Experimental economics
- Survey validation