Complexity in risk elicitation may affect the conclusions: A demonstration using gender differences

Abstract

The Holt and Laury (American Economic Review, 92(5), 1644–1655, 2002) mechanism (HL) is the most widely-used method for eliciting risk preferences in economics. Participants typically make ten decisions with different variance options, with one of these choices randomly chosen for actual payoff. For this mechanism to provide an accurate measure of risk aversion, participants need to understand the choices and give consistent responses. Unfortunately, inconsistent and even dominated choices are often made. Can these mistakes lead to a misrepresentation of economic phenomena? We use gender differences in risk taking to test this question. In contrast to many findings in the literature, HL results typically do not find significant gender differences. We compare the HL approach, where we replicate the lack of significant gender differences, with a simpler presentation of the same choices in which participants make only one of the ten HL decisions; this simpler presentation yields strong gender differences indicating that women are more risk averse than men. We also find gender differences in the consistency of decisions. We believe that the results found in the simpler case are more reflective of underlying preferences, since the task is considerably easier to understand. Our results suggest that the complexity and structure of the risk elicitation mechanism can affect measured risk preferences. The issue of complexity and comprehension is also likely to be present with elicitation mechanisms in other realms of economic preferences.

Appendix 1: Structural model

An additional approach to the analysis allows us to pool all of the data into a single structural model. We assume a constant relative risk aversion utility function (CRRA), and our procedure is to obtain the structural estimates of the model as follows. We assume that subjects have a utility of money M given as $U\left(M|r\right)=\frac{M^{1-r}}{1-r}$, where r denotes the coefficient of relative risk aversion. For each binary choice between gambles, we assume that subjects evaluate the alternatives by making an expected utility calculation, weighting the utility of each outcome, U(M _k | r), by its probability of occurrence p _k , as follows:

$$ E{U}_i=\sum \limits_k\left[{p}_k\times U\left({M}_k|r\right)\right],\forall k=1,2 $$

for each gamble i. If we further denote EU _A as the Option A gamble and EU _B as the Option B gamble, we can construct a probabilistic choice rule, where the likelihood of choosing Option A is given by:

$$ \frac{E{U}_A^{\frac{1}{\mu }}}{E{U}_A^{\frac{1}{\mu }}+E{U}_B^{\frac{1}{\mu }}} $$

The parameter μ allows for deviations from the deterministic choice of the highest expected utility option that is specified by expected utility theory. As in previous work, μ is interpreted as a kind of noise or error: as μ → ∞ the choice becomes a random decision, and as μ → 0 subjects behave exactly as specified by expected utility theory. This parameter can also be thought of as heterogeneity in behavior not captured by the model.

The ratio above forms the basis of a logistic conditional logarithmic likelihood function, denoted as $\mathcal{L}\left(r,\mu |{Y}_i\right)$, which is maximized with respect to r and μ, where the vector Y _i denotes the actual subjects’ choices for either Option A or Option B. In order to allow for heterogeneity by treatment, gender and location, each of the parameters in the vector [r, μ] is specified as a function of these factors, X _i , with associated coefficient vector β. The resulting modified likelihood function is written as $\mathcal{L}\left(r,\mu, \beta |{Y}_i,{X}_i\right)$.

The results of this estimation are shown in Table 8, which pools data from both treatments and provides a useful summary of the results discussed in the main body of the paper. The table reports a model for factors affecting the risk-aversion parameter r and the so-called “noise” parameter μ. The model includes dummy variables indicating the full HL treatment (HL-10), whether the subject is male, and if the sessions were conducted in San Diego, and an interaction between HL-10 and gender. For r, a larger positive coefficient indicates greater risk aversion, and a negative coefficient risk-seeking. For μ, larger coefficients indicate a greater divergence from Expected Utility maximization, showing degree of noise or heterogeneity not captured by the model.

Table 8 Estimates of risk and noise parameters as functions of characteristics

The strongly negative Male estimated coefficient for r shows that males are considerably less risk averse than females in the single-choice treatment. Controlling for the differential effect for men, there is no significant difference in risk aversion for women across the single-choice and HL-10 mechanisms, as shown by the insignificant coefficient on HL-10, consistent with our results in Table 7. However, the large positive coefficient on the HL-10*Male interaction offsets the negative main effect for Male, indicating that the gender difference is not present with the HL-10 risk elicitation, as the sum of the Male and the HL-10*Male coefficients is an insignificant −0.065. Only the single-row treatment shows a strong gender difference. This supports our previous analysis showing a gender difference in the single-choice treatment, but not in the standard HL-10 treatment. There is also no significant difference in risk aversion across location, as indicated by the respective small-to-modest and insignificant coefficients and San Diego.

Turning to the noise parameter μ, the coefficient of the Male dummy is not significant. Furthermore, the negative coefficient of the HL-10*Male interaction dummy wipes out the slightly positive Male coefficient, so that we see no significant difference across gender with respect to noise. The slightly negative coefficient on HL-10 is far from significance. Consistent with prior analysis, we do find that the San Diego data are indeed noisier.

Note that our estimates are not strictly comparable to other structural models using HL data because we have dropped five of the “rows” in the full HL measure in order to focus on the comparison between HL and the single-row measure that we use. Nevertheless, our estimates of the HL-10 CRRA are within the range of those reported in Filippin and Crosetto (2016). The noise estimates are lower, because the dropped rows constitute a “coarser” categorization of subjects who completed the full HL. (Their study controls for differences in instructions for the task, but does not report the estimates.)