When and how to satisfice: an experimental investigation

Abstract

This paper is about satisficing behaviour. Rather tautologically, this is when decision-makers are satisfied with achieving some objective, rather than in obtaining the best outcome. The term was coined by Simon (Q J Econ 69:99–118, 1955), and has stimulated many discussions and theories. Prominent amongst these theories are models of incomplete preferences, models of behaviour under ambiguity, theories of rational inattention, and search theories. Most of these, however, seem to lack an answer to at least one of two key questions: when should the decision-maker (DM) satisfice; and how should the DM satisfice. In a sense, search models answer the latter question (in that the theory tells the DM when to stop searching), but not the former; moreover, usually the question as to whether any search at all is justified is left to a footnote. A recent paper by Manski (Theory Decis. doi:10.1007/s11238-017-9592-1, 2017) fills the gaps in the literature and answers the questions: when and how to satisfice? He achieves this by setting the decision problem in an ambiguous situation (so that probabilities do not exist, and many preference functionals can therefore not be applied) and by using the Minimax Regret criterion as the preference functional. The results are simple and intuitive. This paper reports on an experimental test of his theory. The results show that some of his propositions (those relating to the ‘how’) appear to be empirically valid while others (those relating to the ‘when’) are less so.

1 Introduction

This paper is about satisficing behaviour. Way back in 1955 Herbert Simon made a call for a new kind of economics stating that

the task is to replace the global rationality of economic man with a kind of rational behavior that is compatible with the access to information and the computational capacities that are actually possessed by organisms, including man, in the kinds of environment in which such organisms exist. (p 99)

There is a fundamental conflict here provoked by the use of the word ‘rational’, and economists’ obsession with it. The problem is that the expression ‘rational behaviour’ covers virtually all forms of behaviour, as long as it is motivated by some ‘rational’ objective function, and the decision-maker has all relevant information available to him or to her, and the decision-maker (henceforth, DM) can perform all the necessary calculations costlessly. If calculations are costly, then we are led into the infinite regression problem, first pointed out by Conlisk (1996), and rational behaviour, as defined by economists, cannot exist. We are, therefore, constrained to operate with rational models, defined as above. The way forward, within the economics paradigm, is therefore to weaken our ideas of what we mean by rational behaviour. This is the way that economics has been moving. Prominent amongst these latter weaker theories are theories of incomplete preferences (Ok et al. 2012; Nau 2006; Mandler 2005; Dubra et al. 2004); theories of behaviour under ambiguity (Etner et al. 2012; Gajdos et al. 2008; Ghirardato et al. 2004; Hayashi and Wada 2010; Klibanoff et al. 2005; Schmeidler 1989; Siniscalchi 2009); theories of rational inattention (Sims 2003; Manzini and Mariotti 2014; Matejka and McKay 2015; Caplin and Dean 2015); and search theories (Masatlioglu and Nakajima 2013; McCall 1970; Morgan and Manning 1985; Stigler 1961). A useful survey of satisficing choice procedures can be found in Papi (2012).

Almost definitionally, models of incomplete preferences have to be concerned with satisficing: if the DM does not know his or her preferences, it is clearly impossible to find the best action. These models effectively impose satisficing as the only possible strategy. The problem here is that complete predictions of behaviour must also be impossible. Prediction is possible in models of behaviour under ambiguity. But here again satisficing behaviour ‘must’ occur, if only because not all the relevant information is available to the DM. Unless the DM’s information is objectively correct, there is presumably always some action that is better than the one chosen by the DM. But here the DM does not choose to satisfice; nor does he or she choose how to satisfice. Models of rational inattention also capture the idea of ‘satisficing’ behaviour—in that choice is made from a subset of the set of possible actions—those which capture the attention of the DM, that is, those which are in the consideration set of the DM. However, these theories are silent on the reasons for the formation of a consideration set, and, in some of them, on how the consideration set is formed.

We examine a new theory—that of Manski (2017)—which might be classified as an extended search model. Search models seem to be closest to the scenario in which Manski’s paper is set. Standard search models assume that the DM is searching for the highest number in some distribution and that there is a cost of obtaining a drawing from that distribution. Because of this cost, the DM does not keep on searching until he or she finds the highest number: generally he or she should keep on searching until a ‘sufficiently’ high number is found. This could be termed the DM’s aspiration level. One interpretation of Manski’s paper is that he generalises the story: in addition to being able to search for numbers greater than some (or several) aspiration level(s), the DM can pay a higher search cost and be able to find the highest number, and also the DM can choose not to indulge in any search and simply receive a lower number. Manski not only considers choice between these three strategies, but also the choice of the aspiration level(s). This is the ‘how’ of Manski’s theory: he explains how many times satisficing should be implemented, how aspiration levels should be formed and how they should be changed in the light of the information received.¹

We experimentally test this new theory. Some of the other models that we have discussed have also been tested experimentally; for incomplete preferences we refer the reader to Cettolin and Riedl (2016), Costa-Gomes et al. (2014) and Danan and Ziegelmeyer (2006); for behaviour under ambiguity to Abdellaoui et al. (2011), Ahn et al. (2010), Halevy (2007), Hey and Pace (2014) and Hey et al. (2010); for rational inattention to Chetty et al. (2009), De Los Santos et al. (2012); and for search theories to Caplin et al. (2011), De Los Santos et al. (2012), Hayashi and Wada (2010) and Reutskaja et al. (2011). Our experimental test has some similarities in common with some of these and some differences. In some ways our test is closest to that of Hayashi and Wada (2010), though they test minimax, $\alpha$-maximin and the (linear) contraction model (Gajdos et al. 2008). We test Manski’s model and have a different way of generating imprecise information/ambiguity.

In the next section we describe the Manski model, while in Sect. 3 we discuss the experimental design. Our results are in Sect. 4, and Sect. 5 concludes.

2 Manski’s model of satisficing

In the model the DM has to choose some action. The DM knows that there is a set of actions, each member of the set implying some payoff. The payoffs of these actions are bounded between a lower bound, L, and an upper bound, U, which are known to the DM. Hence, without costly deliberation, the DM faces a problem under ambiguity as he or she does not have sufficient knowledge to determine the optimal decision—that of choosing the action which yields the highest payoff. However, the DM can learn more about the payoff values subject to different costs, which in turn, yield different benefits. There are three available deliberation strategies: ‘No Deliberation’, ‘Satisficing’, and ‘Optimising’. ‘No Deliberation’ incurs no cost and yields only the value of the payoff of an arbitrarily chosen action. ‘Optimising’ has a positive cost (K) and reveals the maximum payoff value. ‘Satisficing’ has a positive cost (k) and provides information whether there are actions that are at least as large as some specified aspiration level.

Crucial to the model is that the assumed objective of the DM is the minimisation of maximum regret (MMR). One reason for this is that there is no known probability distribution of the payoffs, so, for example Expected Utility theory and its various generalisations cannot be applied.² Additionally, and crucially for our experiment, the solution is an ex ante solution, saying what the DM should plan to do as viewed from the beginning of the problem. As Manski writes “I study ex ante minimax-regret (MMR) decision making with commitment”. So the DM is perceived of as choosing a strategy at the beginning of the problem, and then implementing it. This implies a resolute decision-maker. If the DM is not resolute the solution may not be applicable.

The paper applies the ex ante minimax-regret rule to this environment and derives a set of simple, yet intuitive, decision criteria for both the static and the dynamic choice situation. Simon (1955) also suggested that there can be a sequence of deliberations/satisficing where the DM adjusts his or her aspiration level in the light of information discovered. Hence, the dynamic choice situation is of particular interest. Manski’s theory (in his Proposition 2) is that

• (1) The optimal (maximum) number of rounds of deliberation ($M^*)$ if the DM uses a satisficing strategy is given by

  $$\begin{aligned} M^{*}=\hbox {int}\Big [\frac{\log \left( \frac{U-L}{k}\right) }{\log (2)}\Big ] \end{aligned}$$

• (2) If the DM uses a satisficing strategy, the DM sets the aspiration level $t_{m}$ in the $m\hbox {'th}$ round of satisficing as follows:

  $$\begin{aligned} t_m =\frac{L_m +U_m }{2} \end{aligned}$$

  Here $t_{m}$ denotes the aspiration level in round m and $L_{m}$ and $U_{m}$ are the lower and upper bounds on the payoffs given what the DM has observed up to round m.

• (3)

  • (a) Optimisation is an MMR decision if

    $$\begin{aligned} K\le U-L\quad \hbox { and }\quad K\le kM^{*}+\frac{U-L}{2^{{M^*}}} \end{aligned}$$

  • (b) Satisficing with $M^*$ and $t_{m}\, (m=1,{\ldots },M^*)$ is an MMR decision if

    $$\begin{aligned} k\le \frac{U-L}{2}\quad \hbox { and }\quad K\ge kM^{*}+\frac{U-L}{2^{{M^*}}} \end{aligned}$$

  • (c) No Deliberation is an MMR decision if

    $$\begin{aligned} k\ge \frac{U-L}{2}\quad \hbox { and }\quad K\ge U-L \end{aligned}$$

The intuition of the theory is simple. Deliberation costs play a central role. ‘Optimising’ or ‘Satisficing’ will be the decision if their respective associated cost (K, k) is low enough. If both costs are sufficiently large then ‘No Deliberation’ will be preferred. If ‘Satisficing’ is chosen, the aspiration level is midway between the relevant lower bound and the relevant upper bound, while the number of deliberation rounds is decreasing in its associated cost. This theory is different from the existing search literature in that it provides the concept of satisficing search that follows more closely Simon’s perception of adaptive aspiration levels than standard search models. It clearly states when the DM should satisfice. It also provides a solution to the choice of aspiration levels.

Before we move on to the experiment, let us briefly translate the above theory into a description of behaviour. The DM starts with knowing that there is a set of payoffs (the number of them unknown) lying between some lower bound L and some upper bound U. The DM is told the values of k and K. The first thing that the DM needs to do is to design a strategy. This depends on the values of k and K. If these are sufficiently large (see 3c above), the DM decides not to incur these costs and chooses ‘No Deliberation’. The DM is then told and given the payoff of the first action in the choice set, and that is the end of the story.

If K is sufficiently small (see 3a above) the DM decides to incur this cost and ‘Optimise’ and hence learn the highest payoff. He or she gets paid the highest payoff minus K, and that is the end of the story.

The interesting case is 3b, where k is sufficiently small and K sufficiently large. The DM then decides to satisfice with (a maximum³ of) $M^{*}$ rounds (as given by 1 above)⁴ of satisficing. In each of these $M^{*}$ rounds, the DM sets an aspiration level, pays k and is told at the end of the round whether or not there are payoffs greater than or equal to the stated aspiration level. More precisely, the DM is told whether there are 0, 1 or more than 1 payoffs greater than or equal to the stated aspiration level .The DM then updates his or her views about the lower and upper bounds on the payoffs in the light of the information received. This updating procedure is simple:

• If there are no payoffs greater than aspiration level $t_{m}$ then $L_{m+1}= L_{m}$ and $U_{m+1} = t_{m}$

• If there are payoffs greater than aspiration level $t_{m}$ then $L_{m+1}= t_{m}$ and $U_{m+1} = U_{m},$

where $L_{m}$ and $U_{m}$ are the lower and upper bounds after m rounds of satisficing.

When at most $M^{*}$ rounds have been completed the DM gets paid the payoff of the first action in the range between his or her current lower bound and the current upper bound minus kM (the costs of deliberation), where M is the actual number of rounds of satisficing implemented $(M\le M^{*})$.

This paper reports on an experiment to test the theory. We test whether subjects choose between ‘No Deliberation’, ‘Satisficing’ and ‘Optimising’ correctly [as in (3) above]. We also test, when subjects choose to satisfice, whether they choose the correct number of rounds of satisficing [as in (1) above], and whether aspiration levels are chosen correctly [as in (2) above].

3 Experimental design

The actual experimental design differs in certain respects from the design of the theory. First, we told subjects that if they implemented ‘No Deliberation’ they would be paid the lowest payoff in the choice set, rather than the payoff of the first-ordered element of the choice set. Second, we only told subjects, when they chose to satisfice with an aspiration level t, whether there were or were not payoffs greater than or equal to t, and not whether there were 0, 1 or more than 1. Moreover, if after satisficing for m rounds, and discovering that there were payoffs in a set $[L_{m},U_{m}]$, if they chose ‘No (further) Deliberation’ at that point they would get a payoff equal to the lowest payoff in the set $[L_{m},U_{m}]$ minus mk. These differences do not change the predictions of the theory in that an MMR decision-maker will always assume that the first element is the lowest element. Additionally, the ex ante choice of $M^{*}$ remains the same.

Let us give an example (which was included in the Instructions to the subjects). To make this example clear, we need to introduce some notation: the variable lvgeal is defined as the lowest payoff greater than or equal to the highest aspiration level for which there are payoffs greater than or equal to the aspiration level.

On the screen (see the screenshot below) there were three buttons

The one on the left corresponds to ‘No Deliberation’, the one in the middle to ‘Satisfice’ and the one on the right to ‘Optimise’. In this example $k=1$ and $K=10$.

Suppose—though the DM does not know this and our subjects were not told this—that the payoffs are 55   18      75      19      9

If the DM clicks on the left-hand button straight away the income would be 9 (the lowest payoff).

If the DM clicks on the right-hand button straight away the income would be 65 (the highest payoff, 75, minus K).

If the DM clicked on the middle button and specified an aspiration level of 40, he or she would be told that there are payoffs greater than this, but would not be told how many nor what they are. The software would, however, note that the lowest payoff greater than or equal to 40 is 55. This would be the lvgeal defined above. If the DM clicked on the left-hand button at this stage his or her income would be 54 (lvgeal minus k). After this first round of satisficing the DM’s $L_{1}$ and $U_{1}$ are 40 and 100, respectively.

If the DM now clicks on the middle button again and now specifies an aspiration level of 70, he or she would be told that there are payoffs greater than this, but would not be told how many nor what they are. The software would, however, note that the lowest payoff greater than or equal to 70 is 75. This would become the lvgeal. If the DM clicks on the left-hand button at this stage the income for this problem would be 73 (lvgeal minus 2k). After this second round of satisficing the DM’s $L_{2}$ and $U_{2}$ are 70 and 100, respectively.

If the DM now clicks on the middle button a third time, and now specifies an aspiration level of 80, he or she would be told that there are no payoffs greater than this. The software would, however, keep the lvgeal, 75, in memory. If the DM clicks on the left-hand button at this stage the income for this problem would be 72 (lvgeal minus 3k). After this third round of satisficing the DM’s $L_{3}$ and $U_{3}$ are 70 and 80, respectively.

Subjects could keep on clicking on the middle button as often as they wanted, but they were told that the cost would be deducted from the payoff each time.

Note that in this particular case, it is better to click on the middle button twice (with aspiration levels of 40 and 70) and then on the left-hand button, rather than to click on either the left-hand button or the right-hand button straight away, and better than to click on the middle button one or three times (with aspiration levels of 40, 70 and 80) and then on the left-hand button. But this is not always the case.

In the experiment, 48 subjects were sequentially presented with 100 problems on the computer screen, all of the same type. They were given written Instructions and then shown a PowerPoint presentation of the instruction before going on to the main experiment. Subjects were informed of the lower (L) and upper (U) bounds on the payoffs in each problem; these were fixed at 1 and 100, respectively. They were also told the two types of cost: the cost of finding out whether there are any payoffs greater or equal to some specified aspiration level (k) and the cost of finding the highest payoff (K). The number of payoffs (N) was fixed at 5, though subjects were not given this information.⁵ We used the procedure in Stecher et al. (2011) to generate the ambiguous distributed payoffs. This procedure creates complete ambiguity for subjects as they have no way to put any probabilities on the payoffs. To make this clear to the subjects we inserted Online Figures 4 and 5⁶ in the Instructions. Each of them contains 49 distributions, each of 10,000 replications. In Online Figure 4 the drawings were from a uniform distribution over the entire range; in Online Figure 5 from an ambiguous distribution as derived using the Stecher et al. (2011) method. It will be seen that all the distributions in Online Figure 4 are approximately uniform, while those in Online Figure 5 are all completely different. We told the subjects that “this means that one cannot attach probabilities to each of the numbers coming up. Probabilities are undefined.”

Fig. 1

Sets of k and K for Treatment 1 (left) and Treatment 2 (right) plotted in the parameter space

Fig. 2

Partition of the parameter space into areas corresponding to the theoretical predictions

We ran two different treatments, Treatment 1 and Treatment 2. In each of these subjects were presented with 100 problems. In Treatment 1, we had four different values for k and K (with N, L and U fixed across the 100 problems); and we gave the subjects these 4 problems in 4 blocks of 25, with the order of the blocks randomised across subjects. In Treatment 2, we had 100 different values for k and K in each of the 100 problems and presented the problems in a randomised order (again with N, L and U fixed across the 100 problems). Figure 1 illustrates. Figure 2 shows the predictions of the theory.

All 48 subjects completed the experiment which was conducted in the EXEC Lab at the University of York. Subjects’ ages ranged from 18 to 44 years. Educational backgrounds were: high school graduate or equivalent (9 subjects); college credit (8); bachelor degree (19); master degree (11); and professional degree (1). 46 subjects reported themselves as a student (8 subjects in a bachelor degree, 9 subjects in a master degree and 11 subjects in doctoral degree); one subject was a member of staff at the University of York; one subject did not report his/her current degree/position. Subjects’ ethnicities were mainly White (26 subjects) while 18 were Asian/Pacific Islander, 3 were Black or African American and 1 other. There were only 5 subjects who had any work experience related to finance or economics, but most of them (34 subjects) had previously participated in an economics experiment.

To be a fair test of the theory, we need to give incentives to the subjects to act in accordance with it. We should repeat the fact that the theory is an ex ante theory: it tells DMs what to do as viewed from the beginning of a problem; it assumes commitment. Clearly, given the nature of the experiment, we cannot observe what the subjects plan ex ante, nor can we check whether they implement their plan. All we can observe is what they do, so we are testing the theory in its entirety—meaning the validity of all its assumptions.⁷ Ex ante the objective of the theory is to minimise the maximum regret. Ex ante Regret is the difference between the maximum possible income and their actual income. The maximum possible value of the former is exogenous—it depends upon the problem which in our case is always 100 ex ante. So minimising the ex ante maximum regret is achieved by maximising their income. So we paid them their (average⁸) income.

A subject’s payment from the experiment was their average income from all 100 problems plus the show-up fee of £2.50. Average income was expressed in Experimental Currency Units (ECU). Each ECU was worth $33{}^{1}\!/{}_{3}\hbox {p}$; that is 3 ECU was equivalent to £1. They filled in a brief questionnaire after completing all problems on the computer screen, were paid, signed a receipt and were free to go. The average payment was £13.05. This experiment was run using purpose-written software written (mainly by Paolo Crosetto) in Python 2.7.

4 Results and analyses

The purpose of the experiment was to test Proposition 2 of Manski (2017) as stated in Sect. 3. First, we compare the actual and theoretical decisions for all subjects and in each treatment. Second, we compare the actual and theoretical predictions for income and regret. Third, we analyse the number of rounds of satisficing by comparing the theoretical and actual number for all subjects and both treatments. Finally, we analyse the subjects’ actual aspiration levels and compare them with those of the theory.

4.1 When to satisfice

Our experiment gives us 4800 decisions (between ‘No Deliberation’, ‘Satisficing’ and ‘Optimising’) across 48 subjects and 100 problems. Table 1 gives a comparison of the actual and the theoretical decisions; here the main diagonal indicates where subjects followed the theoretical prediction. From this table it can be seen that 2693 out of the 4800 decisions (56.10%) are in agreement with the theoretical. The number of theoretical predictions for each strategy can be found at the end of each row while the total number of subjects’ decisions can be found at the bottom of each column. Subjects appear to choose ‘No Deliberation’ significantly more than the theoretical prediction (49.88% compared with 17.50%). Comparing Treatment 1 with Treatment 2 shows that Treatment 2 is closer to the Manski optimal than Treatment 1: 1476 out of 2400 actual decisions (61.50%) match with the theoretical in Treatment 2 compared to 1217 out of 2400 actual decisions (50.71%) in Treatment 1.⁹

Table 1 Actual vs theoretical decisions for all the subjects

In Table 2 we compare the actual and theoretical average income and average regret. Obviously, it must be the case that actual regret is higher than the theoretical regret (as subjects were not always following the theory). Subjects also have a higher average income. This suggests that subjects may have been working with a different objective function,¹⁰ or making some assumption about the distribution of the payoffs that was not true.¹¹ Comparing the two treatments, we see that subjects in Treatment 2 have relatively better results in terms of the average income (33.40 ECU to 30.10 ECU) and regret (95.20 ECU to 121.10 ECU) than in Treatment 1. This is interesting, as the idea of Treatment 1 (where each problem was repeated 25 times) was to give subjects a chance to learn; we had expected performance to be better there. Perhaps they learnt about the ‘distribution’ of payoffs and therefore departed from the theory?

Table 2 Actual average vs theoretical average for income and regret

Table 3 Actual vs theoretical number of rounds of satisficing

4.2 How to satisfice

Table 3 compares the theoretical (maximum¹²) and the actual number of rounds of satisficing (obviously restricted to the cases where they actually satisficed). There are 452 problems out of 3120 problems (14.49%), where the subjects should satisfice, and where they choose the same number of rounds of deliberation as the theoretical prediction. The difference between treatments is small: 16.67 and 11.89% matches of theoretical and actual number of rounds of deliberation, for treatments 1 and 2 respectively. Generally they choose fewer rounds of satisficing than the theory predicts.¹³

Figure 3 shows a plot of actual vs theoretical aspiration levels for all subjects (and separately for those in Treatments 1 and 2) where the subjects chose to satisfice.¹⁴ We calculate the theoretical aspiration level based on the relevant lower and upper bounds at the time of choosing satisficing. The 45$^{\circ }$ line shows what subjects should do if they select their aspiration level according to the theory. The figure shows that subjects’ aspiration levels increase with the theoretical levels, although the mean equality test shows a rejection of equal means between the actual and theoretical aspiration level when subjects do satisficing (t test $=$ 15.19, $p = 0.000$) for all the subjects. Doing this analysis for each treatment separately shows the same result.

Fig. 3

Actual vs theoretical aspiration level

We now investigate more closely whether subjects set their aspiration level as the theory predicts: equal to the mid-point between the relevant upper and lower bounds. We report below regressions of the actual aspiration level against the optimal level. If the theory holds, the intercept should be zero and the slope should be equal to 1. We omit observations where the aspiration level was above the upper bound (see footnote 9), and accordingly, carry out truncated regressions. Before we proceed to the regressions, we note that the correlations between the actual and theoretical aspiration level 0.544 over all subjects, 0.513 for Treatment 1 and 0.569 for Treatment 2.

Table 4 shows that the coefficient on the theoretical aspiration level is not significantly different from 1 in Model 1. However, in Model 1 we have included a constant term which should not be there; unfortunately, it is significantly different from 0, which it should not be. If we remove the constant term to get Model 2, we find that the slope coefficient is almost significantly different from 1. So this table tells us that subjects are almost but not quite following the Manski’s rule.

We broke down the analysis of Table 4 by treatments. The results are similar for Model 1 in both treatments. In Model 2, we find that the slope coefficient is significantly different from 1 in both treatments.

We now delve deeper and try to understand how the actual aspiration levels are determined, and in particular, how they are related to the upper and lower bounds. We present below regressions of the subjects’ aspiration level as a function of these bounds. If following the theory the relationship should be $AL_{im} =0.5L_{im} + 0.5U_{im}$ (where $AL_{im}$ is subject i’s aspiration level in round m of satisficing and $L_{im}$ and $U_{im}$ are the relevant lower and upper bounds). As before, we have excluded outliers (aspiration levels greater than the upper bound) from the regression and performed truncated regressions.

Table 4 Regressions of the actual aspiration level on the theoretical aspiration level for all subjects

Table 5 Regressions of the actual aspiration level on the lower and upper bounds for all subjects

Table 5, over all the subjects, shows that the estimated parameters on the bounds are significantly different from the theoretical value of 0.5 and that the subjects put more weight on the upper bound and less on the lower bound when they select their aspiration levels.

If we break down the analysis of Table 5 by treatments, we see some differences between them. In Treatment 1 the estimated parameters are significantly different from the theoretical 0.5 (with more weight put on the upper bound than the lower), while in Treatment 2 they are much closer (and indeed only significantly different from 0.5 for one estimated parameter). So in Treatment 2 the subjects are closer to the theory in this respect than in Treatment 1. This confirms an earlier result. Possibly it was a consequence of the fact that in Treatment 2 each problem was an entirely new one, while in Treatment 1 (where 4 problems were given in blocks of 25) subjects were ‘learning’ about the distribution of payoffs¹⁵ and thus departing from the theory: as the subjects were working through the 25 problems they felt that they were getting some information about the ‘distribution’.

5 Conclusions

The overall conclusion must be that subjects were not following the part of the theory regarding the ‘when’ question: the choice between ‘No Deliberation’, ‘Satisficing’ and ‘Optimising’, possibly as a consequence of our experimental design.¹⁶ However, the choice of the number of rounds of satisficing is closer to the theory. The first of these is a particularly difficult task and the second slightly less difficult, and, therefore, these results may not be surprising. In addition, subjects may have experienced difficulties in understanding what was meant by an ambiguous distribution. However, when it comes to the choice of the aspiration levels, subjects are generally close to (though sometimes statistically significant from) the optimal choice of $(L+U)/2$. This latter task is easier and more intuitive. So it seems that the ‘when’ part of the theory is not empirically validated, while part of the ‘how’ part receives more empirical support.

One serious problem with our experimental test (which we have already mentioned) is that the theory is an ex ante theory, and one with commitment (so the DM is resolute), while our experimental test involves observing what subjects actually do. A full ex ante test is difficult as we would have to ask subjects to specify, not only their choice of deliberation strategy, but also their choice of conditional aspiration levels. Perhaps we could go part-way there by getting the computer to implement some stated aspiration levels, telling subjects the computer algorithm, and asking subjects simply to choose between ‘No Deliberation’, ‘Satisficing’ and ‘Optimisation’. This would be a partial test—one answering only the ‘when’ of the title. Other variations are possible, but all appear to be difficult.

Let us restate that the theory is an ex ante theory and one with commitment: the DM is committed to his or her ex ante plan and implements it resolutely. The theoretical predictions may be different if the DM is not resolute. Let us illustrate this with the choice of $M^{*}$. At the beginning of the problem the DM calculates $M^*$—which depends on L and U at the beginning. After m rounds of satisficing the DM will have updated lower and upper bounds. Suppose he or she re-calculates the relevant $M^*$—call this $M_{m}^{*}$. Will it be true that $M_{m}^*$ is equal to $M^*-m$? We see no reason why that should be so—it depends upon the information that the DM has acquired. So it seems perfectly reasonable that a DM should revise his or her plan as he or she works through a problem. But then this is not the optimal way to solve the problem even if the DM is a MMR agent—backward induction should be employed. Perhaps this is what our subjects were doing?

In conclusion, we should note that there are three crucial elements to the theory: the use of the MMR preference functional, commitment and the perception of the payoffs as having an ambiguous ‘distribution’. The violation of any of these would lead to a breakdown of the theory. We tried to ensure that subjects perceived the ‘distribution’ as being ambiguous in our experiment. We tried to incentivise the use of the MMR preference functional by our payment rule, but the subjects could well have had a different objective function.¹⁷ Unfortunately, it seems difficult to force commitment on the subjects, and they may well have been revising their strategy as they were working through a problem. Nevertheless, subjects seem to have been following the theory in at least one key respect—the choice of their aspiration levels.