From what we wear to how we entertain ourselves, we are constantly choosing among sets of alternatives. It is as if our lives are merely extended department store experiences, where we commonly make hundreds of choices before we reach the checkout aisle. Although these decisions often rely on past experience—for which there is ample literature (Hertwig, Barron, Weber, & Erev, 2004; Hertwig & Erev, 2009; Rakow, Demes, & Newell, 2008; Ungemach, Chater, & Stewart, 2009)—two important qualities separate them from what has been the dominant focus of prior research. First, our choices often involve more than two options—sometimes ranging into the hundreds (consider cheeses in French supermarkets)—and therefore potentially lead to choice-set size effects that are predicted to involve information overload (Iyengar & Lepper, 2000; Mick, Broniarczyk, & Haidt, 2004; Scheibehenne, Greifeneder, & Todd, 2010; Schwartz, 2004). Second, our choices are often sequential, with the order of choice-set sizes varying from one decision to the next (e.g., there may be more kinds of cheese than bananas). As a result, the order of choice sets may lead to choice-set size effects that carry over from one decision to the next (Levav, Heitmann, Herrmann, & Iyengar, 2010; Levav, Reinholtz, & Lin, 2012). This points to the need for a more fine-grained and practically relevant study of information processing under choice overload.

Here we investigated the influences of choice-set size and choice order effects on information search and information processing. In particular, we focused on the prospect that information overload may influence search, choice, and the role of infrequent, risky events. This combines two previously independent literatures on consumer choices over multiple options and decisions from experience. Before describing our study in more detail, we will briefly review the relevant findings from these two literatures.

Information overload and choice order effects

Choices with many options have been suggested to lead to choice overload. As examples, choices over many options have been shown to lead to forgoing decisions or choosing simpler options (Chernev, 2003; Greifeneder, Scheibehenne, & Kleber, 2010; Iyengar & Kamenica, 2010; Iyengar & Lepper, 2000; Reutskaja & Hogarth, 2009; Schwartz, 2004; Shah & Wolford, 2007). In addition to arguments based on regret and satisfaction (e.g., Iyengar & Lepper, 2000; Schwartz, 2004), the predominant explanation for choice overload effects has been information overload (e.g., Jacoby, Speller, & Kohn, 1974; Lee & Lee, 2004; Schwartz, 2004). As options increase in number, efforts to make rational decisions may succumb to the overwhelming amount of information processing required. In such cases, too many options may influence information search, information processing, or both. Indeed, choice overload effects may represent a cognitive limit on models of choice developed for small set sizes.

However, the claims regarding choice overload have been equivocal. For example, an attempt to replicate the jam study of Iyengar and Lepper (2000) revealed no evidence of choice overload (Scheibehenne, 2008). In a more recent meta-analysis, Scheibehenne et al. (2010) found that the cross-study effect size, based on more than 60 studies of choice overload, was “virtually zero.” In addition to the studies in the meta-analysis, still further studies have continued to find inconsistent support for choice overload (e.g., Arunachalam, Henneberry, Lusk, & Norwood, 2009; Bunford & Szrek, 2010). These conflicting results call for a more detailed analysis of the information processing underlying choice over many options.

In addition to choice overload, choice-set size can also influence behavior through potential carryover effects. Recent work on choice order effects has shown that varying the order of choice environments—for instance, making choices among many options before choices among fewer options—can influence choices even after set sizes have decreased (Levav et al., 2010; Levav et al., 2012). For example, Levav et al. (2012) found that when people chose songs among varying sizes of music selections, they listened to fewer songs before choosing one when set sizes varied from many to few songs than when set sizes varied from few to many. Similarly, in a naturalistic study on car buyers, buyers who saw vehicle options (e.g., engine types and gearshift styles) were more likely to choose the default option if they experienced choices with many (as opposed to few) alternatives first (Levav et al., 2010).

Taken together, the results above suggest the consequences of large choice sets potentially lead to information overload and may, in addition, carry over to smaller choice sets. Unfortunately, finding evidence for information overload at the level of information processing has been difficult, because previous research in this area has dominantly looked at choices among items without objective measures of quality (e.g., songs, pens, and chocolates). Subjective choice criteria have also made it difficult to examine other aspects of choices, such as expected value and risk. To answer such questions requires a quantitative decision paradigm.

Decisions from experience

The sampling paradigm, used in research on decisions from experience, provides a well-tested method for independently studying information search, information processing, and subsequent decision making (Hertwig & Erev, 2009). Here, we refer to our paradigm of interest as the sequential-sampling paradigm in order to distinguish it from other sampling paradigms (e.g., Hilbig & Glöckner, 2011). With the sequential-sampling paradigm, people sample ad lib from two gambles with initially unknown payoff distributions—exploring them—before choosing the gamble that will determine their final payoff. Sampling involves selecting a gamble and witnessing an outcome sampled from its payoff distribution. For example, one distribution could offer $32 with probability .10, or 0 otherwise, and the other distribution could offer $3 for sure. Sampling might then lead a participant to experience 0, 0, 32, 0 in the first distribution, and 3, 3, 3 in the second, before making a final decision. Thus, as in the jam study of Iyengar and Lepper (2000), participants can sample items as much as they like and then choose the one that they most prefer. However, the sequential-sampling paradigm allows us to investigate not only how decision environments influence information search and subsequent choice, but also how the outcomes of the search influence the subsequent decisions (e.g., Hau, Pleskac, Kiefer, & Hertwig, 2008).

Though studied dominantly with set sizes of two, the sequential-sampling paradigm can be scaled to include more gambles available for sampling. This is the approach that we have taken here. Nonetheless, even with two samples, the sequential-sampling paradigm provides several observations relevant to our present investigation. First, information search foreshadows decisions (Hau et al., 2008; Hills & Hertwig, 2010). In particular, people who sample more and switch less often between gambles are more likely to choose gambles with higher expected values. Because choice order effects may lead to more sampling for individuals who sample few items first (Levav et al., 2012), it follows that these individuals may also be more likely to choose gambles with higher expected values.

How sampling scales to large choice-set sizes is not clear, and leads to the second insight from previous work on the sequential-sampling paradigm. In decisions from experience, people make choices that suggest that they may underweight rare events (Hau et al., 2008; Hertwig et al., 2004; Hertwig & Erev, 2009; Ungemach et al., 2009). When presented with two gambles, people often choose as if rare events occur less frequently than their true probability. In part, this is explained by small samples (Hertwig et al., 2004; Rakow et al., 2008), with individuals simply failing to experience rare events due to a paucity of sampling. In other words, the apparent underweighting is not always necessarily a result of biased weighting of the observed outcomes, but may reflect proper weighting of small samples. How would this influence choices when set sizes are large? In Levav et al. (2012), increasing set sizes of songs led to less information being sampled per song, but more songs being sampled in total. If this were to be the case in the sequential-sampling paradigm with many gambles, we could predict that participants would be more likely to experience a rare event, because they would sample more risky gambles. However, they would have sampled less from any individual gamble. Thus, with large set sizes, rare events will be encountered not because of intensive sampling with a single gamble—which would provide good information about the expected value—but because of distributed sampling over many gambles. In part, this would amplify the differences between gambles—as predicted by the amplification effect (Hertwig & Pleskac, 2008)—but, importantly for large set sizes, the gambles with the highest sample mean will often be the riskiest gambles—what we call search-amplified risk.

The present study

The present study addressed the issues above using a simple manipulation of the sequential-sampling paradigm: changing the order of choice-set sizes from few to many (2, 4, 8, 16, and 32 gambles) or from many to few (32, 16, 8, 4, and 2 gambles). Our primary concerns were how choice-set size and choice order would influence information search, information processing, and choice quality. In this study, we investigated three predictions that follow directly from the research outlined in the introduction: (1) Choice order and choice-set size effects would influence information search, leading, respectively, to more search in the few-to-many condition than in the many-to-few condition, as well as more samples overall but fewer samples per gamble as set sizes increased. (2) Differences in information search induced by choice order effects would lead to choices for gambles with higher expected values in the few-to-many than in the many-to-few condition, especially when set sizes were small. The final prediction depended on the influence of information overload: (3) If the theory of information overload applies in the sequential-sampling paradigm with large choice sets, this would lead to choices less in line with expected value maximization as choice sets grow larger, due to failures of information processing; on the other hand, if the theory of information overload does not apply—with decisions for large choice sets appearing to rely on the same information as do decisions for small choice sets—we predicted that search-amplified risk would lead to choices favoring riskier gambles.

Method

The participants were 64 undergraduates from the University of Warwick, who received course credit as well as monetary compensation (ranging from £1 to £5) based on their choices. Participants sat in front of a computer and saw the following instructions:

Each box is a money machine that pays off a certain amount with a certain probability. You can sample these boxes as many times as you want to try and determine which box you want to choose to receive your payoff. During the sampling phase, you will not receive any payoffs. When you decide to make a final decision, you will get one draw from this gamble “for real” as your payoff for that set of gambles.

Following these instructions, participants saw an array of boxes with the instruction to “sample the options until you are ready to choose one.” Clicking on a box caused an outcome from its probability distribution to be sampled and presented for 2 s. Participants continued sampling until they indicated that they were ready to make a final decision. Then they were instructed to “choose the option from which you would like to receive a final payoff” and allowed to click on one of the boxes from the same array. The participants were randomly assigned to either the many-to-few or the few-to-many condition, representing choice-set sizes of 32, 16, 8, 4, and 2 gambles, experienced in that order or in the opposite order, respectively. Thus, we employed a 2 (condition: few-to-many or many-to-few, between subjects) × 5 (set size: 32, 16, 8, 4, and 2 gambles, within subjects) mixed design.

The payoffs within a choice set were constructed as follows. An initial value, h, was randomly sampled for each gamble from a distribution with mean 5 and standard deviation 1. Then payoff probabilities were assigned each gamble by randomly assigning values (without replacement) from an equal-spaced grid, with grid size being proportional to the number of gambles in the choice set: for two gambles probabilities were .67 and .33; for four gambles, .2, .4, .6, and .8; and so on. Finally, the initial value, h, was divided by the probability in order to produce the nonzero payoff, rounded to the nearest integer. For example, a gamble with an h of 5.21 and a probability of .2 would pay off 26 with probability .2, or otherwise 0. Note that because of the way that the choices were created (trading off maximum outcome with probability of payoff), no single choice would dominate the others on maximum value, median value, and expected value simultaneously. The participants in both conditions sampled from the same payoff distributions for each set size.

Results

The impact on information search

Both choice order and choice-set size influenced information search for a variety of measures (Fig. 1). Participants took more samples with larger set sizes and more samples in the few-to-many than in the many-to-few condition (Fig. 1a). Predicting total sample size, we found main effects of set size [χ2(1) = 83.63, p < .001] and condition [χ2(1) = 5.41, p = .020].Footnote 1 The proportions of gambles sampled were also affected (Fig. 1b): Proportionally fewer gambles were sampled as set size increased [χ 2(1) = 24.75, p < .001], and participants in the few-to-many condition sampling proportionally more gambles than did those in the many-to-few condition [χ 2(1) = 5.87, p = .015].

Fig. 1
figure 1

Influences of choice-set size and choice order on information search: a total samples, b proportions of gambles sampled, c samples per gamble, and d switches per sample. Error bars are 95 % confidence intervals

Full size image

Participants in the few-to-many condition also sampled each gamble more times, despite sampling from more gambles overall (Fig. 1c). Results predicting samples per gamble showed significant main effects of set size [χ 2(1) = 49.58, p < .001] and condition [χ 2(1) = 5.13, p = .024], as well as a significant interaction [χ 2(1) = 17.80, p < .001]. As is shown in Fig. 1c, both conditions became more similar as set sizes increased, suggesting that as choice-set size increased, the influence of large set sizes overshadowed choice order effects. Finally, participants tended to switch between gambles less frequently per sample in the few-to-many than in the many-to-few condition, though this effect was not significant at a threshold of p < .05 [Fig. 1d; χ 2(1) = 3.74, p = .053]. However, both conditions switched between gambles more often as set sizes increased [χ 2(1) = 54.37, p < .001]. Taken together, the results above strongly support our prediction that choice order and choice-set size have systematic influences on information search.

The impact on choice value

Did the changes in information search also influence participants’ choices with respect to the expected value? To evaluate this question, we computed for each decision the relative rank of the expected value for the chosen gamble. Here we defined the relative rank as being equal to (n − r)/(n − 1), where r is the rank of the chosen gamble and n is the choice-set size. Figure 2a shows influences of both choice order and set size on the relative rank of the expected value for chosen gambles.Footnote 2 The results of a beta regression including both condition and log set size showed a nonsignificant effect for set size (b = 0.11, p = .068), a significant effect for condition (b = 0.74 p < .001), and a significant interaction between condition and set size (b = −0.26, p = .001). Taking each condition individually, the few-to-many condition showed a significant effect of set size (b = −0.16, p = .005), whereas the many-to-few condition did not (b = 0.11, p = .073). In comparison with random choice (relative rank = .5), one-sample t tests indicated that only the few-to-many condition differed significantly, and only when as few as two [t(30) = 5.04, p < .001] or four [t(31) = 3.14, p < .001] gambles were presented. For both conditions, with larger set sizes the probability of choosing gambles with ranks higher than .5 was no different than would be expected from random choices (using a Bonferroni-corrected criterion of p > .01).

Fig. 2
figure 2

Influences of choice-set size and choice order on the expected value and sample mean of chosen gambles. a Relative ranks of expected values. b Relative ranks of sample means. The dotted lines indicate the expected rank if choices were random with respect to the vertical axis. Error bars are 95 % confidence intervals

Full size image

In sum, choice order influenced decision quality with respect to the expected value, as we predicted. In particular, those in the few-to-many condition chose gambles with higher expected values, consistent with their increased information search. However, during or after making decisions among large set sizes, participants were no better than random at choosing gambles with higher expected values.

The impact on information processing

Our last prediction depended on whether or not information overload influenced information processing. To evaluate information processing, we used choice models to predict choices on the basis of outcomes of the information search phase (e.g., Hau et al., 2008). Note that the experiment was not designed to discriminate among the best choice models, as each participant only made one choice for each set size, and samples per gamble were small for the largest set sizes. Thus, the most well-supported models from prior research were indistinguishable. Instead, our experiment was designed to evaluate whether or not information processing becomes more susceptible to noise with large set sizes, as predicted by the information overload hypothesis. Here, we present only a representative model from the best performing of those that we tried: the sample mean (i.e., the natural mean or mean of the experienced outcomes for a gamble; see Hau et al., 2008).Footnote 3

To provide a measure of how well outcomes predicted choice, we ranked gambles by the relative means of their outcomes. Figure 2b shows the relative ranks of the chosen gambles, with respect to the sample mean, as proportions of the ranks of the other sampled gambles—that is, (s − r)/(s − 1), where r is the rank of the chosen gamble and s is the number of sampled gambles. The results of a beta regression including both choice order and choice-set size showed that the two conditions did not differ in their sensitivities to the sample mean (b = 0.27, p = .111), but were influenced by the set size (b = 0.12, p = .019), with higher-ranked gambles being chosen for larger set sizes. Because the conditions were not different, we aggregated them in order to evaluate whether or not the relative ranks were significantly above .5. The results of one-sample t tests comparing the aggregated relative rank with .5 for each set size consistently had p < .001. Because participants chose higher-ranked gambles as the number of gambles increased, this indicated that choices became even more systematic in favor of the sample mean as choice-set size increased. This result indicates that choices did not become noisier with increased set sizes, as predicted by the information overload hypothesis (Schwartz, 2004), but instead appear to have become even more consistent with the sample mean (possibly due to the amplification effect; Hertwig & Pleskac, 2008).

Search-amplified risk: The impact on risky choice

The results above show effects on information search and choice, but they also reveal that choices did not become less systematic with respect to the information acquired for large set sizes. As we predicted in the introduction, in the absence of information overload, the observed changes in information search—that is, more total sampling but fewer samples per gamble—should lead to search-amplified risk effects. To investigate this, we examined a measure of riskiness—the coefficient of variation (CV)—of the chosen gambles (Weber, Shafir, & Blais, 2004). The results of a beta regression including choice order and set size showed a significant effect of condition (b = −1.17, p < .001; Fig. 3a), no effect of set size (b = 0.05, p = .58), and a significant interaction (b = 0.38, p = .003). When taken individually, only the few-to-many condition showed a significant increase in CV (b = 0.27, p < .001), whereas the many-to-few condition did not (b = 0.04, p = .510). However, this result mainly reflected the lower CVs for the highest-expected-value options available in the smaller choice sets. Looking at the choice-set sizes for which this was not the case (i.e., eight or more options), a beta regression revealed no relationship between a participant’s sample mean ranking (e.g., Fig. 2b) and the probability that choices would return a zero payoff (i.e., riskier choices) for choices with eight options (b = 0.04, p = .92), but it did show positive relationships for choice-set sizes of 16 (b = 1.62, p = .002) and 32 (b = 1.10, p = .043). Moreover, for participants who chose options with sample mean rankings greater than .5 in set sizes of 16 and 32, their choices returned zero payoffs 56 % of the time, which was significantly above a chance level of 50 % [results of a one-sample t test averaging first within participants: t(25) = −2.17, p = .039]. This supports our prediction of increased risk-taking among users of the sample mean when they were confronted with the largest set sizes.

Fig. 3
figure 3

Influences of choice-set size and choice order on the risk qualities of chosen gambles. a Coefficients of variation of the chosen gambles. b Sample means and expected values of the chosen gambles. Error bars are 95 % confidence intervals

Full size image

Did search-amplified risk contribute to the increase in risky choice for large set sizes? This would be reflected by a disproportionate increase in the sample mean of chosen gambles with respect to their underlying expected value—indicating an influence of rare events. Figure 3b shows that as set sizes increased, the sample means of the chosen gambles increased, with a significant main effect of set size [χ 2(1) = 26.35, p < .001; neither the condition effect nor the interaction was significant, p > .1]. However, the expected value of the chosen gambles did not change for either condition [χ 2(1) < 2, p > .2]. This indicates that rare events drove this increase in the sample mean—consistent with search-amplified risk. In other words, as set sizes increased, participants were more likely to sample rare positive events and, for some individuals, this led them to choose these riskier gambles.

Discussion

By combining approaches from consumer choice over many options and decisions from experience, the present research has made contributions to both domains. Foremost, choice-set size and choice order influenced information search (as in Levav et al., 2010; Levav et al., 2012), with effects on the total samples taken, proportions of gambles sampled, samples per gamble, and the propensity to switch between gambles. Information processing, on the other hand, appeared to scale across set sizes, contradicting one of the core explanations for choice overload (discussed in Scheibehenne et al., 2010). Decisions were predicted by the outcomes of information search, with participants consistently choosing gambles associated with higher sample means. Thus, the influences on decisions caused by changes in choice-set size and choice order appear to be dominantly mediated by changes in information search. In turn, changes in information search led to systematic changes in risky choice, with participants tending to make more choices for gambles with rare events as set sizes increased.

Why do we not find an effect of information overload? One possibility is that in the sequential-sampling paradigm participants can control how much information they sample. Thus, as predicted by Jacoby (1984; see also Malhotra, 1984), individuals may make procedurally rational decisions (Simon, 1978) and limit search, to avoid overwhelming their cognitive capacities. This capacity to control information is, of course, also possible in many of the choice overload paradigms, and this may explain the absence of a consistent strong effect found in the meta-analysis by Scheibehenne et al. (2010). In the present case, individuals made rational and consistent decisions based on the outcomes of their information search, even for large choice sets. This may indicate that choice overload—when it occurs—is not a consequence of changes in information processing, but is instead driven by the meta-cognitive realization that, with many options, things are not always what they seem. If people are aware of the sampling biases they create with large choice sets (e.g., and thus anticipate regret; Loomes & Sugden, 1982), this may explain the observations associated with choice overload, including hesitation and choosing less complex options (Iyengar & Lepper, 2000; Mick et al., 2004).

Despite consistent information processing, changes in information search had dramatic effects on risky choice as set sizes increased, with participants tending to choose gambles with rarer positive outcomes. This is in contrast to the prior research on decisions from experience, which have been interpreted as indicating an underweighting of rare events (e.g., Hertwig et al., 2004; Hills & Hertwig, 2010; Rakow et al., 2008; Ungemach et al., 2009). As we noted in the introduction, the impact of rare events as set sizes increase is a predictable outcome of certain changes in information search. With large set sizes, increased sampling overall increases the likelihood of encountering a rare event. However, reduced sampling per gamble reduces the opportunity for these events to be encountered alongside more common outcomes from that distribution. This both amplifies the differences in sample means between gambles (Hertwig & Pleskac, 2008) and leads to search-amplified risk—with risky gambles being the ones most likely to have the highest sample means. Thus, the greater likelihood of choosing gambles with rare events may not be due to a bias in the processing of sampled outcomes, but to a bias in sampling.

Though this apparent overweighting of rare events appears at first glance to be at odds with previous research (Hau et al., 2008; Hertwig et al., 2004; Ungemach et al., 2009), it is nonetheless a consequence of the same phenomenon of limited search scaled to many options. In practice, with large set sizes, people may often make the trade-off between information overload and search-amplified risk—risking either too much information to process or information being biased in favor of risky choices. Future research will be needed to address this possibility, as well as how best to deal with it.

Finally, what may be behind the observed carryover effects in information search? We can offer two possibilities. First, people may adapt to one search environment by tuning an implicit cognitive search parameter (e.g., giving-up times), and then transfer this parameter to subsequent search environments (Hills, Todd, & Goldstone, 2010). In the present case, this might mean that giving-up times were tuned to the initial choice environments with few or many options, and then, when choice-set sizes changed, participants did not update these parameters. Alternatively, set size may influence explicit maximizing or satisficing mindsets, which then transfer to new environments (Levav et al., 2012). This might lead people to believe that they can make optimal choices, even in complex environments, because they have made similar choices in less complex environments. Though our research was not designed to distinguish between these two options, the question of the implicit or explicit nature of the observed carryover effects remains open for future research. As we found significant differences in people’s likelihoods of choosing gambles with higher expected values when set sizes were small, understanding how choice order and set size influence decision making has clear practical implications for real-world decisions, be they consumer or otherwise.