1 Introduction

Expected Utility Theory (EUT) is one of the oldest and widely used criteria for decision making under risk. Bernoulli (1738) first proposed EUT to resolve the St. Petersburg Paradox. Von Neumann and Morgenstern (1947) provided a normatively appealing axiomatization of EUT. Yet, Allais (1953) challenged the descriptive accuracy of EUT with two examples. His first example (Allais, 1953, p. 527) is known as the Allais Paradox or common-consequence effect. Following MacCrimmon and Larsson (1979), his second example (Allais, 1953, p. 529) is known as the common-ratio effect. One illustration of this effect, due to Kahneman and Tversky (1979, p. 266), is that subjects choose $3,000 for sure over an 80% chance of winning $4,000 but choose a 20% chance of $4,000 over a 25% chance of $3,000 (when probabilities are scaled down by the same ratio, which gives the effect its name).

A wide-spread perception is that the common-ratio effect is a robust behavioral regularity. More than 40 years ago MacCrimmon and Larsson (1979, Sect. 5.3, p. 369) concluded that “even though the common-consequence problem … is generally called the ‘Allais Paradox’, the [common-ratio effect] which is also due to Allais is more of a ‘paradox’ (if either is) in the sense that it elicits a higher rate of violation of the [EUT] axioms.” Given that the effect has since been tested in numerous studies, we can verify its experimental robustness. Our methodology is similar in spirit to meta-studies: we re-analyze experimental data collected in previous studies (meta-studies re-analyze previously published statistics).

Blavatskyy et al. (2022) re-analyzed data from 29 articles (81 experimental designs/ parameterizations, 8,947 observations) on the Allais Paradox and found that the standard paradox was recorded in 38 designs (46.9%), no paradox was recorded in 27 designs (33.3%) and the reverse paradox was recorded in the remaining 16 designs (19.8%). Here we reexamine data from 39 articles with 143 experimental parameterizations of the common-ratio effect (14,909 observations). Out of these 143 designs, the standard common-ratio effect was found in 85 designs (59.4%), no common- ratio effect was found in 43 designs (30.1%), and the reverse common-ratio effect was found in the remaining 15 designs (10.5%).

Our econometric analysis shows that the common-ratio effect is susceptible to similar effects of experimental design, implementation, and parameter choices as the Allais Paradox. Specific choices of an experimenter can make the common-ratio effect appear, disappear, or reverse. It is important to raise awareness of such effects to promote the development of appropriate non-expected utility theories that can rationalize these effects.

The remainder of the paper is organized as follows. In Sect. 2, we briefly explain the common-ratio effect. In Sect. 3, we explain how we collected the data. In Sect. 4, we provide our regression analysis. In Sect. 5, we offer a concluding discussion.

2 The common-ratio effect

The original Allais (1953, p. 529) example of the common-ratio effect is the following: A decision maker can prefer ₣100 million for certain over 98% chance of ₣500 million (and a 2% chance of nothing) and, at the same time, she can prefer 0.98% chance of ₣500 million (and a 99.02% chance of nothing) over a 1% chance of ₣100 million (and 99% chance of nothing). In the second binary choice problem, the chances of positive gains are scaled down from the corresponding chances in the first binary choice problem by a common ratio of 1/100. This gives the effect its name.

The common-ratio effect is illustrated in the probability triangle (Machina, 1982) in Fig. 1 (axes not to scale). Point A (0,0) represents the prospect of ₣100 million for certain. Point B (0.02, 0.98) represents a 98% chance of ₣500 million. Point C (0.99,0) represents a 1% chance of ₣100 million. Point D (0.9902, 0.0098) represents a 0.98% chance of ₣500 million. Due to the scaling by a common ratio, line AB is parallel to line CD. The left panel of Fig. 1 shows a typical family of indifference curves for an expected-utility maximizer—positively-sloped parallel straight lines. Since AB is parallel to CD, A is located on a higher indifference curve than B (as shown on the left panel of Fig. 1) if and only if C is located on a higher indifference curve than D. Thus, an expected-utility maximizer weakly prefers A over B if and only if she weakly prefers C over D (a risk-averse choice pattern consistent with EUT).

Fig. 1
figure 1

Illustration of the common-ratio effect in the probability triangle (not to scale)

Full size image

A choice of A over B and D over C violates EUT (except for a special case of indifference) and we refer to this choice pattern as the common-ratio pattern. The right panel of Fig. 1 illustrates the common-ratio choice pattern. A decision maker who chooses B over A and C over D likewise violates EUT and we call this the reverse common-ratio pattern. Typically, the majority of decision makers reveal the common-ratio choice pattern and only a minority reveal the reverse common-ratio pattern. It is these behavioural regularities that became known as the common-ratio effect.

Several design, implementation, and parameter choices could affect the likelihood of observing the common-ratio effect. First, a small common ratio reduces the difference in expected utility between lotteries in the second binary choice (“scaled-down” lotteries C and D). Loomes (2005, p. 305) argued that this may increase certain instances of the common-ratio effect when decision makers are prone to Fechner (1860/1966)-type random errors. Such errors are more likely in the second binary choice between “scaled-down” lotteries C and D, which differ little in expected utility, compared to the first binary choice between “scaled-up” lotteries A and B, which differ considerably in expected utility.

Second, the slope of lines AB and CD in the probability triangle might influence the likelihood of the common-ratio effect. Note that the slope of lines AB and CD in the probability triangle is an increasing function of the probability of the highest outcome in risky lottery B in the first common-ratio question. Loomes and Sugden (1998) experimented with several different slopes and found that parameterizations with higher slopes generally reveal higher instances of the common-ratio effect. Parameterizations with higher slopes create a similarity (or inconsequentiality) of probabilities in the second common-ratio problem (the probability of a non-zero outcome in lottery D becomes relatively similar to that in lottery C). This similarity (or inconsequentiality) can catalyze the common-ratio effect. Blavatskyy et al. (2020) reanalyzed experimental data from 81 experiments reported in 29 studies of the Allais Paradox and found that a relatively low slope in the probability triangle is likely to reverse the Allais Paradox. We test if this conclusion also holds for the common-ratio effect.

Third, instances of the common-ratio effect may be affected by the sheer size of monetary outcomes. Lotteries with larger outcomes may induce more risk-averse choices. Ceteris paribus, this increases the likelihood of EUT-consistent risk-averse choice pattern A≿B and C≿D.

Fourth, Blavatskyy (2010, experiment 2, pp. 232–5) found that the common-ratio effect not only disappears but is reversed when the medium outcome is moved away from the highest outcome. A parameterization with a relatively high ratio of middle to highest outcome may increase cognitive load making both common-ratio questions a harder decision problem, which might lead to a higher rate of EUT violations. Blavatskyy et al. (2020) found that the Allais Paradox is more likely in experiments with the medium outcome being close to the highest outcome.

Fifth, instances of the common-ratio effect may be affected by the nature of the incentives used in experimental studies. Debate is ongoing about the effects of hypothetical and real incentives in experimental economics. On the one hand, Kahneman and Tversky (1979, p. 265) argued in favor of hypothetical incentives in experiments on individual choice under risk. On the other hand, financial incentives are the default choice in most economics experiments and there are good reasons for it (e.g., Hertwig & Ortmann, 2001; Ortmann, 2016). Hertwig and Ortmann (2001) argue that the choice of hypothetical or real incentives ought to be evidence-based.

Sixth, Carlin (1992) found that the common-ratio effect is less likely when choice alternatives are represented as compound lotteries or in a frequency format rather than as simple probability distributions. In their meta-study, Blavatskyy et al. (2020) found such an effect for the Allais Paradox. To summarize, we identify six design and implementation details and parameter choices that might drive the results of experimental studies on the common-ratio effect: 1) common ratio itself; 2) slope of lines AB and CD in the probability triangle; 3) size of payoffs; 4) ratio of the middle to the highest outcome; 5) whether incentives are hypothetical or real; and 6) presentation of choice alternatives.

3 Data

We searched in the Scopus database with a string ( TITLE-ABS-KEY ( "common ratio") AND TITLE-ABS-KEY ( "experiment")) excluding non-economic articles, theoretical articles, and experimental studies that collect no data on the common-ratio effect (e.g., Andreoni & Sprenger, 2012; Kelsey & Schepanski, 1991; Müller-Trede et al., 2018). To this list, we added articles identified in the process of data collection for our previous study (Blavatskyy et al. 2020) when in effect the authors collected experimental data on the common-ratio effect (some papers refer to the common-ratio problem as the Allais paradox, e.g., van de Kuilen & Wakker, 2006; Herrmann et al., 2017). We discarded several experimental studies with unconventional parameterizationsFootnote 1 as well as between-subject testsFootnote 2 of the common ratio. Raw experimental data of Kahneman and Tversky (1979) are lost and we cannot infer the frequencies of four choice patterns revealed in their common-ratio experiment. Starmer and Sugden (1989b) use the same experimental data as Starmer and Sugden (1989a). Birnbaum et al. (2017) re-analyze data of Birnbaum and Schmidt (2015). This results in a list of 24 articles. Going through the references in these articles we identified another eight studies that collect data on the common-ratio effect. After circulating the first draft of our working paper we received feedback about seven additional recent studies collecting data on the common-ratio effect. With this final addition, we obtained 39 articles (preceded by an asterisk in the list of references). Data collected from these 39 articles (143 design/parameterizations with 14,909 revealed-choice patterns) are presented in Table 1.

Table 1 Experimental data on the common-ratio effect
Full size table

In the first column of Table 1 we list the different experimental designs/parameterizations. In column 2 we enumerate the number of corresponding participants. Columns 3–6 of Table 1 list how many subjects revealed the four possible choice patterns (risk-averse and risk-seeking EUT-consistent choice patterns, common-ratio and reverse common-ratio choice patterns). Conlisk (1989) developed a test statistic, known as Conlisk z-statistic, which takes values close to zero under the null-hypothesis of there being no EUT violations. Large positive values of the statistic indicate that common-ratio choice patterns outnumber reverse common-ratio choice patterns. Large negative values of the statistic indicate the opposite (reverse common-ratio choice patterns outnumber common-ratio choice patterns). Columns 7 and 8 of Table 1 list Conlisk z-statistic and its corresponding p-value for each of 143 design/parameterizations.

The remaining columns of Table 1 show the realizations of the six design and implementation characteristics identified in the previous section as potentially relevant for instances of the common-ratio effect. Column 9 lists the probability of the highest outcome in risky lottery (B) in the first question (which is an increasing function of the slope of lines AB and CD in the probability triangle). Column 10 lists the common-ratio factor. Column 11 lists the highest outcome and its currency. Column 12 lists the ratio of middle to highest outcome. Column 13 is a dummy variable that equals one (zero) if the researchers used real (hypothetical) incentives. Column 14 shows whether choice alternatives were presented as simple lotteries (1), in frequency form (0*), or as compound lotteries (0). The last column is a dummy variable equal to one if experimental subjects were students.

The sources from which we collected data reported in Table 1 are listed in Table 2 (for 39 experimental articles). We do not consider data from stages 2 and 3 in the experiment of Baillon et al. (2016), repetitions 2–4 in Birnbaum and Schmidt (2015), stages 2 and 3 in the experiment of Bone et al. (1999), repetition 2 in Loomes and Sugden (1998) and repetitions 2–3 in Schmidt and Neugebauer (2007) to avoid any confounding with the learning effects. We do not consider household-decision making in Bateman and Munro (2005) to focus only on individual choice. We do not consider the repeated-play condition in Barron and Erev (2003) or the 10-play and 100-play conditions in DeKay et al. (2016) to focus only on single realization of lotteries. Buschena and Zilberman (1999, p. 261) report that 39% of subjects reveal common-ratio choice pattern in questions 1 and 2 but their Table 4 (p. 270) implies that this percentage is at least 55%-8% = 47%. We assume a typo and the percentage of common-ratio choice patterns in their experiment is taken to be 49%.

Table 2 Sources of experimental data
Full size table

Figure 2 shows the fractions of the observed outcomes of choice patterns across all the experiments in the dataset depending on whether incentives are hypothetical or real, large or small, that is above or below the corresponding median payoff in 2010 USD.Footnote 3 The EUT-consistent risk-averse (AC) pattern is most prevalent in case of real large payoffs, whereas common-ratio (AD) pattern appears in the case of hypothetical large payoff. Interestingly, small hypothetical and real payoffs exhibit similar and closer to uniformly-distributed choice patterns.

Fig. 2
figure 2

Frequency of choice patterns for each of four payoff categories—hypothetical payoffs, large and small, i.e., above and below the median hypothetical payoff of $123 (in 2010 USD) and real payoffs, large and small, relatively to the median real payoff of $34 (in 2010 USD)

Full size image

4 Regression Analysis

In every experiment each subject can reveal only one of four possible choice patterns: risk-averse EUT-consistent choice, risk-seeking EUT-consistent choice, common-ratio choice pattern, and reverse common-ratio choice pattern. In this case, a multinomial logistic specification is appropriate:

$$\begin {array} {*{20}ll} \ln \left( {\frac{{{ }P_{i} }}{{P_{{\text{RA EUT}}} }}} \right) &= \beta_{i0} + \beta_{i1} \ln {\mathbf{P}} \times {\mathbf{I}} + \beta_{i2} \ln {\mathbf{P}} \times \left( {1 - {\mathbf{I}}} \right) \\& + \beta_{i3} {\mathbf{I}} + \beta_{i4} {\mathbf{L}} + \beta_{i5} {\mathbf{S}} + \beta_{i6} {\mathbf{O}} + \beta_{i7} {\mathbf{PH}} + \beta_{i8} {\mathbf{CR}},\end {array}$$

where \({P}_{i}\) is the probability of observing a specific choice pattern, i = {risk-seeking EUT, common-ratio, and reverse common-ratio} and risk-averse EUT-consistent choice pattern is set as the baseline outcome and explanatory variables are:

  • highest outcome \({\mathbf{P}}\) (column 11 of Table 1, after conversion to 2010 USD, see footnote 3);

  • real-incentives dummy variable \({\mathbf{I}}\) (column 13 of Table 1);

  • probability of the highest outcome in a risky lottery in the first common-ratio questionFootnote 4\({\mathbf{PH}}\) (column 9 of Table 1); and

  • the ratio of middle to highest outcome \({\mathbf{O}}\) (column 12 of Table 1);

  • lottery presentation categorical variable \({\mathbf{L}}\) (column 14 of Table 1)Footnote 5;

  • student dummy variable S (column 15 of Table 1)Footnote 6;

  • the common ratio \({\mathbf{CR}}\) (column 10 of Table 1).

The highest outcomes \({\mathbf{P}}\) are natural-logged to reconcile a wide range of their values and to reflect saturation. There is a strong negative correlation between \(\ln {\mathbf{P}}\) and the real incentives dummy variable \({\mathbf{I}}\), as studies with high payoffs use only hypothetical incentives. We use the interaction terms \(\ln {\mathbf{P}} \times {\mathbf{I}}\) and \(\ln {\mathbf{P}} \times \left( {1 - {\mathbf{I}}} \right)\) to allow for different slopes for \(\ln {\mathbf{P}}\user2{ }\) for the cases of real and hypothetical payoffs, respectively. We also consider several alternative model specifications presented in Table 4 in the supplementary material. (They produce similar results with a lower goodness of fit.)

Table 3 presents the average marginal effects (observation-specific marginal effects averaged over all observations) of the 4-outcome logistic regression (regression coefficients \(\beta\) are presented in Table 5 in the supplementary material). Note that average marginal effects for each explanatory variable sum up to zero over the four possible choice patterns. Table 3 reports regular standard errors as well as cluster-robust standard errors. The cluster-robust method allows for correlated residuals within clusters, but not across clusters. Correlations may be induced by some unobserved conditions specific to a cluster. We cluster at the level of the country resulting in 11 clusters.

Table 3 Average marginal effects computed from the logit model
Full size table

Looking at the probability of observing the common-ratio choice pattern (the section of Table 3 highlighted in boxed text), we find that it is influenced by the following factors. First, a lower common ratio (the last column of Table 3) increases the probability that subjects reveal the common-ratio choice pattern by 0.054 for each 0.1 decrease in the common ratio largely because it decreases the likelihood that subjects reveal the risk-averse EUT-consistent choice. For instance, if we reduce the overall median common ratio of 0.25 to 0.01, as in the original Allais (1953, p. 529) example, the probability of observing the common-ratio choice pattern increases by approximately 0.13. This confirms the well-known observation that experiments with low common ratios produce high instances of EUT violations. We could also speculate that this fact is probably well-known because it happens to be the strongest factor influencing the likelihood of the common-ratio effect.

Second, conducting an experiment with hypothetical incentives increases the chance of observing the common-ratio choice pattern by 0.011 for each relative increase in payoffs (the third column of Table 3). In combination with the intercept term shifter (the fourth column) that is actually lower for the hypothetical incentives than for the real incentives, we conclude that very high hypothetical incentives lead to the increased probability of the common-ratio choice pattern relatively to the low hypothetical incentives. Conversely, for real incentives, the chance of observing the common-ratio choice pattern is reduced by 0.054 for each relative increase in real payoffs. Hence, high real incentives lead to the reduced observations of the common-ratio choice pattern in comparison to low real incentives. This is largely due to the fact that high real incentives increase the likelihood of the EUT-consistent choices (both risk-averse and risk-seeking).

Third, subjects are more likely to reveal the common-ratio effect when choice alternatives are described as simple probability distributions (not in a compound or frequency form), cf. the seventh column of Table 3. Largely this happens because subjects are then less likely to reveal the risk-seeking EUT-consistent choice pattern. For example, changing the presentation format from compound to simple lotteries increases the probability of observing the common-ratio effect by 0.212.

Fourth, designing the common-ratio experiment with the middle outcome being close to the highest outcome (cf. the sixth column of Table 3) favours the instances of the common-ratio choice pattern at the expense of EUT consistent choices and the reverse common-ratio choice pattern. Blavatskyy et al. (2020, Table 2) report a similar effect for the Allais paradox.

Based on our findings we can construct a common-ratio index capturing the main features of the experimental design that favor the common ratio effectFootnote 7:

$${\text{CRindex}} = - 0.054{ }\ln {\mathbf{P}} \times {\mathbf{I}} + 0.011 \ln {\mathbf{P}} \times \left( {1 - {\mathbf{I}}} \right) + 0.244{ }{\mathbf{I}} + 0.147{\mathbf{O}} + 0.212{ }{\mathbf{L}} + 0.537 {\mathbf{CR}}$$

Figure 3 plots this index against the share of subjects revealing the common-ratio choice pattern in each of the 143 experimental designs in our data set. The index has predictive abilities.

Fig. 3
figure 3

Share of subjects revealing the common-ratio choice pattern vs. common-ratio index. N = 143 experimental designs. Linear trend is indicated by the dotted line and its equation is reported

Full size image

5 Discussion

Over the last three decades researchers have produced large quantities of data on numerous behavioral regularities. It is important to take stock as experimental results are sometimes contradictory and one can fail to see the forest for the trees, possibly misdirecting theory efforts. In this paper we have reexamined experimental data on one specific behavioral regularity in choice under risk — the common-ratio effect. Our methodology is similar to meta-studies that sample previously published results in a systematic and replicable manner except that we re-analyze previously collected experimental data rather than previously published results.

We believe that our approach limits the publication bias and selective reporting that may be present in traditional meta-studies (cf. Kvarven et al., 2020). We re-analyze experimental data from articles published on a variety of topics, not necessarily focusing on the common-ratio effect. For example, Agranov and Ortoleva (2017) study a preference for deliberate randomization in stochastic choice; Baillon et al. (2016) and Bone et al. (1999) compare group and individual decision making; Bateman and Munro (2005) study decision making in households; Battalio et al. (1990) and Harless and Camerer (1994) compare the goodness of fit of different non-EUT theories; Buschena and Zilberman (1999) study the effects of similarity; Blondel et al. (2007) study preferences of drug addicts; Butler and Loomes (2011) study imprecision of preferences under risk; Chetty et al. (2020) investigate the trust game; Harrison et al. (2018) study smoking behavior; Fatas et al. (2007) and Linde and Vis (2017) study decision making of politicians; Hey and DiCagno (1990) elicit indifference curves in the probability triangle; Loomes and Sugden (1998) compare the goodness of fit of different models of probabilistic choice; Schmidt and Neugebauer (2007) study the effect of random errors on risky choice; Wu (1994) studies ordinal independence of preferences. Given that the primary focus of these studies is not the common-ratio effect per se (which appears in their experimental treatments by serendipity) there is no ex ante reason to expect any publication bias or selective reporting with respect to the common-ratio effect in these studies.

Examining a large body of empirical evidence on the common-ratio effect during the last 40 years shows some remarkable regularities. Some of these regularities are well-known. For example, the fact that experimental designs with a small common ratio induce more instances of the common-ratio choice pattern is built into many descriptive decision theories such as rank-dependent utility (Quiggin, 1981) or cumulative prospect theory (Tversky & Kahneman, 1992) that use inverse S-shaped probability weighting function to capture this effect. Other regularities we documented above are less known if not outright surprising.

For example, we find that the common-ratio effect is more likely to be observed when choice alternatives are presented as simple probability distributions, i.e. not described in frequency or compound-lottery format. We also find that common-ratio experiments with very high hypothetical incentives are more likely to document the common-ratio choice pattern. If we subscribe to the point of view that substantially high real incentives reduce the impact of random errors, noise, or imprecision in revealed preferences (Hertwig & Ortmann, 2001), then our results suggest that the EUT risk-averse outcome is a behavioral regularity that is more likely to be observed once randomness and imprecision in revealed preferences are reduced.

Another less known finding is that common-ratio experiments with middle lottery outcome being close to the highest lottery outcome are more likely to document the common-ratio choice pattern. This effect is consistent with findings in Blavatskyy (2010, experiment 2, pp. 232–5) that the common-ratio effect gets reversed when the middle outcome is moved away from the highest outcome. Testing the common-ratio effect with lotteries that have the middle outcome close to the highest outcome induces similarity (Rubinstein, 1988) in the second pairwise choice between scaled-down lotteries C and D. This could induce a higher likelihood of observing the common-ratio choice pattern.

The standard common-ratio effect was found in 85 out of 143 (59.4%) experimental designs analyzed in this paper. This could be interpreted as a large effect revealing strong evidence against expected utility theory. We do not disagree with this interpretation of the data but reserve our judgement. Our results indicate several experimental design and implementation choices that could affect the likelihood of the common-ratio effect. If the existing literature largely used designs favoring the standard common-ratio effect, then it is hardly surprising that the experimental results are problematic for expected utility theory. Yet, had the literature largely used other designs identified in our paper, then the experimental results would have been different. This suggest that a systematic exploration of the whole parameter space would be desirable.

It is important to raise awareness of how different behavioral regularities are affected by the design and implementation characteristics and parameter choices of experimenters. For empirical work, our findings are important for the design of future experiments. For theoretical work, our findings provide guidelines for the development of generalized non-expected utility theories. A good descriptive decision theory should not aim at capturing one canonical version of the common-ratio effect. In the spirit of Erev et al. (2017), it should be flexible enough to rationalize instances of the common-ratio effect in some experimental designs/parameterizations but not others.