How common is the common-ratio effect?

The common-ratio effect and the Allais Paradox (common-consequence effect) are the two best‐known violations of Expected Utility Theory. We reexamine data from 39 articles reporting experiments (143 designs/parameterizations, 14,909 observations) and find that the common-ratio effect is systematically affected by experimental design and implementation choices. The common-ratio effect is more likely to be observed in experiments with a low common-ratio factor, a high ratio of middle to highest outcome, when lotteries are presented as simple probability distributions (not in a compound/frequency form), and with high hypothetical incentives.


Introduction
Several design, implementation, and parameter choices could affect the likelihood of observing the common-ratio effect. First, a small common ratio reduces the difference in expected utility between lotteries in the second binary choice ("scaled-down" lotteries C and D). Loomes (2005, p. 305) argued that this may increase certain instances of the common-ratio effect when decision makers are prone to Fechner (1860Fechner ( /1966)-type random errors. Such errors are more likely in the second binary choice between "scaled-down" lotteries C and D, which differ little in expected utility, compared to the first binary choice between "scaled-up" lotteries A and B, which differ considerably in expected utility.
Second, the slope of lines AB and CD in the probability triangle might influence the likelihood of the common-ratio effect. Note that the slope of lines AB and CD in the probability triangle is an increasing function of the probability of the highest outcome in risky lottery B in the first common-ratio question. Loomes and Sugden (1998) experimented with several different slopes and found that parameterizations with higher slopes generally reveal higher instances of the common-ratio effect. Parameterizations with higher slopes create a similarity (or inconsequentiality) of probabilities in the second common-ratio problem (the probability of a non-zero outcome in lottery D becomes relatively similar to that in lottery C). This similarity (or inconsequentiality) can catalyze the commonratio effect. Blavatskyy et al. (2020) reanalyzed experimental data from 81 experiments reported in 29 studies of the Allais Paradox and found that a relatively low slope in the probability triangle is likely to reverse the Allais Paradox. We test if this conclusion also holds for the common-ratio effect.
Third, instances of the common-ratio effect may be affected by the sheer size of monetary outcomes. Lotteries with larger outcomes may induce more risk-averse choices. Ceteris paribus, this increases the likelihood of EUT-consistent risk-averse choice pattern A≿B and C≿D.
Fourth, Blavatskyy (2010, experiment 2, pp. 232-5) found that the common-ratio effect not only disappears but is reversed when the medium outcome is moved away from the highest outcome. A parameterization with a relatively high ratio of middle to highest outcome may increase cognitive load making both common-ratio questions a harder decision problem, which might lead to a higher rate of EUT violations. Blavatskyy et al. (2020) found that the Allais Paradox is more likely in experiments with the medium outcome being close to the highest outcome.
Fifth, instances of the common-ratio effect may be affected by the nature of the incentives used in experimental studies. Debate is ongoing about the effects of hypothetical and real incentives in experimental economics. On the one hand, Kahneman and Tversky (1979, p. 265) argued in favor of hypothetical incentives in experiments on individual choice under risk. On the other hand, financial incentives are the default choice in most economics experiments and there are good reasons for it (e.g., Hertwig & Ortmann, 2001;Ortmann, 2016). Hertwig and Ortmann (2001) argue that the choice of hypothetical or real incentives ought to be evidence-based.
Sixth, Carlin (1992) found that the common-ratio effect is less likely when choice alternatives are represented as compound lotteries or in a frequency format rather than as simple probability distributions. In their meta-study, Blavatskyy et al. (2020) found such an effect for the Allais Paradox. To summarize, we identify six design and implementation details and parameter choices that might drive the results of experimental studies on the common-ratio effect: 1) common ratio itself; 2) slope of lines AB and CD in the probability triangle; 3) size of payoffs; 4) ratio of the middle to the highest outcome; 5) whether incentives are hypothetical or real; and 6) presentation of choice alternatives.

Data
We searched in the Scopus database with a string ( TITLE-ABS-KEY ( "common ratio") AND TITLE-ABS-KEY ( "experiment")) excluding non-economic articles, theoretical articles, and experimental studies that collect no data on the commonratio effect (e.g., Andreoni & Sprenger, 2012;Kelsey & Schepanski, 1991;Müller-Trede et al., 2018). To this list, we added articles identified in the process of data collection for our previous study (Blavatskyy et al. 2020) when in effect the authors collected experimental data on the common-ratio effect (some papers refer to the common-ratio problem as the Allais paradox, e.g., van de Kuilen & Wakker, 2006;Herrmann et al., 2017). We discarded several experimental studies with unconventional parameterizations 1 as well as between-subject tests 2 of the common ratio. Raw experimental data of Kahneman and Tversky (1979) are lost and we cannot infer the frequencies of four choice patterns revealed in their common-ratio experiment. Starmer and Sugden (1989b) use the same experimental data as Starmer and Sugden (1989a). Birnbaum et al. (2017) re-analyze data of Birnbaum and Schmidt (2015). This results in a list of 24 articles. Going through the references in these articles we identified another eight studies that collect data on the common-ratio effect. After circulating the first draft of our working paper we received feedback about seven additional recent studies collecting data on the common-ratio effect. With this final addition, we obtained 39 articles (preceded by an asterisk in the list of references).
Data collected from these 39 articles (143 design/parameterizations with 14,909 revealed-choice patterns) are presented in Table 1.
In the first column of Table 1 we list the different experimental designs/parameterizations. In column 2 we enumerate the number of corresponding participants. Columns 3-6 of Table 1 list how many subjects revealed the four possible choice patterns (risk-averse and risk-seeking EUT-consistent choice patterns, commonratio and reverse common-ratio choice patterns). Conlisk (1989) developed a test statistic, known as Conlisk z-statistic, which takes values close to zero under the null-hypothesis of there being no EUT violations. Large positive values of the statistic indicate that common-ratio choice patterns outnumber reverse common-ratio choice patterns. Large negative values of the statistic indicate the opposite (reverse common-ratio choice patterns outnumber common-ratio choice patterns). Columns 7 and 8 of Table 1 list Conlisk z-statistic and its corresponding p-value for each of 143 design/parameterizations. The remaining columns of Table 1 show the realizations of the six design and implementation characteristics identified in the previous section as potentially relevant for instances of the common-ratio effect. Column 9 lists the probability of the highest outcome in risky lottery (B) in the first question (which is an increasing function of the slope of lines AB and CD in the probability triangle). Column 10 lists the common-ratio factor. Column 11 lists the highest outcome and its currency. Column 12 lists the ratio of middle to highest outcome. Column 13 is a dummy variable that equals one (zero) if the researchers used real (hypothetical) incentives. Column 14 shows whether choice alternatives were presented as simple lotteries (1), in frequency form (0*), or as compound lotteries (0). The last column is a dummy variable equal to one if experimental subjects were students.
The sources from which we collected data reported in Table 1 are listed in Table 2 (for 39 experimental articles). We do not consider data from stages 2 and 3 in the experiment of Baillon et al. (2016), repetitions 2-4 in Birnbaum and Schmidt (2015), stages 2 and 3 in the experiment of Bone et al. (1999), repetition 2 in Loomes and Sugden (1998) and repetitions 2-3 in Schmidt and Neugebauer (2007) to avoid any confounding with the learning effects. We do not consider household-decision making in Bateman and Munro (2005) to focus only on individual choice. We do not consider the repeated-play condition in Barron and Erev (2003) or the 10-play and 100-play conditions in DeKay et al. (2016) to focus only on single realization of lotteries. Buschena and Zilberman (1999, p. 261) report that 39% of subjects reveal common-ratio choice pattern in questions 1 and 2 but their Table 4 (p. 270) implies that this percentage is at least 55%-8% = 47%. We assume a typo and the percentage of common-ratio choice patterns in their experiment is taken to be 49%. Figure 2 shows the fractions of the observed outcomes of choice patterns across all the experiments in the dataset depending on whether incentives are hypothetical or real, large or small, that is above or below the corresponding median payoff in 2010 USD. 3 The EUT-consistent risk-averse (AC) pattern is most prevalent in 1 3 How common is the common-ratio effect?  (2003) (1986)  How common is the common-ratio effect?  (1987) (1989) (1994), C5A-C5B 206 120 6 74 6 8.94 0 0.8 0.25 4000 USD 0.75 0 1 1 a Three randomly chosen subjects were rewarded via random-loƩery incenƟve scheme b excluding five subjects who revealed indifference in common-raƟo quesƟons 43 and 18 c excluding three subjects who revealed indifference in common-raƟo quesƟons 29 and 31 d excluding one subject who revealed indifference in common-raƟo quesƟons e excluding four subjects who revealed indifference in common-raƟo quesƟon 2 f excluding two subjects who revealed indifference in common-ratio quesƟon 22 * frequency and compound loƩery formats are coded both as 0 in the regression analysis case of real large payoffs, whereas common-ratio (AD) pattern appears in the case of hypothetical large payoff. Interestingly, small hypothetical and real payoffs exhibit similar and closer to uniformly-distributed choice patterns.

Regression Analysis
In every experiment each subject can reveal only one of four possible choice patterns: risk-averse EUT-consistent choice, risk-seeking EUT-consistent choice, common-ratio choice pattern, and reverse common-ratio choice pattern. In this case, a multinomial logistic specification is appropriate: where P i is the probability of observing a specific choice pattern, i = {risk-seeking EUT, common-ratio, and reverse common-ratio} and risk-averse EUT-consistent choice pattern is set as the baseline outcome and explanatory variables are: • highest outcome (column 11 of Table 1, after conversion to 2010 USD, see footnote 3); • real-incentives dummy variable (column 13 of Table 1); How common is the common-ratio effect?  Barron and Erev (2003) Barron and Erev (2003, p. 220, p. 225),  Table 1).
The highest outcomes are natural-logged to reconcile a wide range of their values and to reflect saturation. There is a strong negative correlation between ln and the real incentives dummy variable , as studies with high payoffs use only hypothetical incentives. We use the interaction terms ln × and ln × (1 − ) to allow for different slopes for ln for the cases of real and hypothetical payoffs, respectively. We also consider several alternative model specifications presented in Table 4 in the supplementary material. (They produce similar results with a lower goodness of fit.) Table 3 presents the average marginal effects (observation-specific marginal effects averaged over all observations) of the 4-outcome logistic regression (regression coefficients are presented in Table 5 in the supplementary material). Note that average marginal effects for each explanatory variable sum up to zero over the four possible choice patterns. Table 3 reports regular standard errors as well as clusterrobust standard errors. The cluster-robust method allows for correlated residuals within clusters, but not across clusters. Correlations may be induced by some unobserved conditions specific to a cluster. We cluster at the level of the country resulting in 11 clusters.

3
How common is the common-ratio effect? Table 3 Average marginal effects computed from the logit model We report regular standard errors (in parenthesis), and the cluster-robust standard errors (clustered at the country level). Coefficients significant at 0.05 level for both the regular and cluster-robust methods are highlighted with bold black font. Coefficients significant at 0.05 level for the regular, but not the cluster-robust method are highlighted with italics. (Same convention for Tables 4, 5  Looking at the probability of observing the common-ratio choice pattern (the section of Table 3 highlighted in boxed text), we find that it is influenced by the following factors. First, a lower common ratio (the last column of Table 3) increases the probability that subjects reveal the common-ratio choice pattern by 0.054 for each 0.1 decrease in the common ratio largely because it decreases the likelihood that subjects reveal the risk-averse EUT-consistent choice. For instance, if we reduce the overall median common ratio of 0.25 to 0.01, as in the original Allais (1953, p. 529) example, the probability of observing the common-ratio choice pattern increases by approximately 0.13. This confirms the well-known observation that experiments with low common ratios produce high instances of EUT violations. We could also speculate that this fact is probably well-known because it happens to be the strongest factor influencing the likelihood of the common-ratio effect.
Second, conducting an experiment with hypothetical incentives increases the chance of observing the common-ratio choice pattern by 0.011 for each relative increase in payoffs (the third column of Table 3). In combination with the intercept term shifter (the fourth column) that is actually lower for the hypothetical incentives than for the real incentives, we conclude that very high hypothetical incentives lead to the increased probability of the common-ratio choice pattern relatively to the low hypothetical incentives. Conversely, for real incentives, the chance of observing the common-ratio choice pattern is reduced by 0.054 for each relative increase in real payoffs. Hence, high real incentives lead to the reduced observations of the common-ratio choice pattern in comparison to low real incentives. This is largely due to the fact that high real incentives increase the likelihood of the EUT-consistent choices (both risk-averse and risk-seeking).
Third, subjects are more likely to reveal the common-ratio effect when choice alternatives are described as simple probability distributions (not in a compound or frequency form), cf. the seventh column of Table 3. Largely this happens because subjects are then less likely to reveal the risk-seeking EUT-consistent choice pattern. For example, changing the presentation format from compound to simple lotteries increases the probability of observing the common-ratio effect by 0.212.
Fourth, designing the common-ratio experiment with the middle outcome being close to the highest outcome (cf. the sixth column of Table 3) favours the instances of the common-ratio choice pattern at the expense of EUT consistent choices and the reverse common-ratio choice pattern. Blavatskyy et al. (2020, Table 2) report a similar effect for the Allais paradox.
Based on our findings we can construct a common-ratio index capturing the main features of the experimental design that favor the common ratio effect 7 : Figure 3 plots this index against the share of subjects revealing the common-ratio choice pattern in each of the 143 experimental designs in our data set. The index has predictive abilities. CRindex = −0.054 ln × + 0.011 ln × (1 − ) + 0.244 + 0.147 + 0.212 + 0.537 7 We thank a referee for this journal for the suggestion.

3
How common is the common-ratio effect?

Discussion
Over the last three decades researchers have produced large quantities of data on numerous behavioral regularities. It is important to take stock as experimental results are sometimes contradictory and one can fail to see the forest for the trees, possibly misdirecting theory efforts. In this paper we have reexamined experimental data on one specific behavioral regularity in choice under risk -the common-ratio effect. Our methodology is similar to meta-studies that sample previously published results in a systematic and replicable manner except that we re-analyze previously collected experimental data rather than previously published results.
We believe that our approach limits the publication bias and selective reporting that may be present in traditional meta-studies (cf. Kvarven et al., 2020). We reanalyze experimental data from articles published on a variety of topics, not necessarily focusing on the common-ratio effect. For example, Agranov and Ortoleva (2017) study a preference for deliberate randomization in stochastic choice; Baillon et al. (2016) and Bone et al. (1999) compare group and individual decision making; Bateman and Munro (2005) study decision making in households;  and Harless and Camerer (1994) compare the goodness of fit of different non-EUT theories; Buschena and Zilberman (1999) Fatas et al. (2007) and Linde and Vis (2017) study decision making of politicians; Hey and DiCagno (1990) elicit indifference curves in the probability triangle; Loomes and Sugden (1998) compare the goodness of fit of different models of probabilistic choice; Schmidt and Neugebauer (2007) study the effect of random errors on risky choice; Wu (1994) studies ordinal independence of preferences. Given that the primary focus of these studies is not the common-ratio effect per se (which appears in their experimental treatments y = 1.1941x R² = 0.0101 Fig. 3 Share of subjects revealing the common-ratio choice pattern vs. common-ratio index. N = 143 experimental designs. Linear trend is indicated by the dotted line and its equation is reported by serendipity) there is no ex ante reason to expect any publication bias or selective reporting with respect to the common-ratio effect in these studies.
Examining a large body of empirical evidence on the common-ratio effect during the last 40 years shows some remarkable regularities. Some of these regularities are well-known. For example, the fact that experimental designs with a small common ratio induce more instances of the common-ratio choice pattern is built into many descriptive decision theories such as rank-dependent utility (Quiggin, 1981) or cumulative prospect theory (Tversky & Kahneman, 1992) that use inverse S-shaped probability weighting function to capture this effect. Other regularities we documented above are less known if not outright surprising.
For example, we find that the common-ratio effect is more likely to be observed when choice alternatives are presented as simple probability distributions, i.e. not described in frequency or compound-lottery format. We also find that common-ratio experiments with very high hypothetical incentives are more likely to document the common-ratio choice pattern. If we subscribe to the point of view that substantially high real incentives reduce the impact of random errors, noise, or imprecision in revealed preferences (Hertwig & Ortmann, 2001), then our results suggest that the EUT risk-averse outcome is a behavioral regularity that is more likely to be observed once randomness and imprecision in revealed preferences are reduced.
Another less known finding is that common-ratio experiments with middle lottery outcome being close to the highest lottery outcome are more likely to document the common-ratio choice pattern. This effect is consistent with findings in Blavatskyy (2010, experiment 2, pp. 232-5) that the common-ratio effect gets reversed when the middle outcome is moved away from the highest outcome. Testing the common-ratio effect with lotteries that have the middle outcome close to the highest outcome induces similarity (Rubinstein, 1988) in the second pairwise choice between scaled-down lotteries C and D. This could induce a higher likelihood of observing the common-ratio choice pattern.
The standard common-ratio effect was found in 85 out of 143 (59.4%) experimental designs analyzed in this paper. This could be interpreted as a large effect revealing strong evidence against expected utility theory. We do not disagree with this interpretation of the data but reserve our judgement. Our results indicate several experimental design and implementation choices that could affect the likelihood of the commonratio effect. If the existing literature largely used designs favoring the standard common-ratio effect, then it is hardly surprising that the experimental results are problematic for expected utility theory. Yet, had the literature largely used other designs identified in our paper, then the experimental results would have been different. This suggest that a systematic exploration of the whole parameter space would be desirable.
It is important to raise awareness of how different behavioral regularities are affected by the design and implementation characteristics and parameter choices of experimenters. For empirical work, our findings are important for the design of future experiments. For theoretical work, our findings provide guidelines for the development of generalized non-expected utility theories. A good descriptive decision theory should not aim at capturing one canonical version of the common-ratio effect. In the spirit of Erev et al. (2017), it should be flexible enough to rationalize instances of the common-ratio effect in some experimental designs/parameterizations but not others.