When and How to Satisfice An Experimental Investigation

This paper is about satisficing behaviour. Rather tautologically, this is when decision-makers are satisfied with achieving some objective, rather than in obtaining the best outcome. The term was coined by Herbert Simon in 1955, and has stimulated many discussions and theories. Prominent amongst these theories are models of incomplete preferences, models of behaviour under ambiguity, theories of rational inattention, and search theories. Most of these, however, seem to lack an answer to at least one of two key questions: when should the decision-maker (DM) satisfice; and how should the DM satisfice. In a sense, search models answer the latter question (in that the theory tells the DM when to stop searching), but not the former; moreover, usually the question as to whether any search at all is justified is left to a footnote. A recent paper by Manski (2017) fills the gaps in the literature and answers the questions: when and how to satisfice? He achieves this by setting the decision problem in an ambiguous situation (so that probabilities do not exist, and many preference functionals can therefore not be applied) and by using the Minimax Regret criterion as the preference functional. The results are simple and intuitive. This paper reports on an experimental test of his theory. The results show that some of his propositions (those (cid:396)(cid:286)(cid:367)(cid:258)(cid:410)(cid:349)(cid:374)(cid:336)


INTRODUCTION
This paper is about satisficing behaviour. Way back in 1955 Herbert Simon made a call for a new kind of economics stating that: behavior that is compatible with the access to information and the computational capacities that are actually possessed by organisms, including man, in the kinds of envi (p 99) There is a fundamental conflict here provoked by the use of the word obsession with it. The problem maker has all relevant information available to him or to her, and the decision-maker (henceforth, DM) can perform all the necessary calculations costlessly. If calculations are costly, then we are led into the infinite regression problem, first pointed out by Conlisk in 1996, and rational behaviour, as defined by economists, cannot exist. We are therefore constrained to operate with rational models, defined as above. The way forward, within the economics paradigm, is therefore to weaken our ideas of what we mean by rational behaviour. This is the way that economics has been moving. Prominent amongst these latter weaker theories are theories of incomplete preferences  (2015)); and search theories (Masatlioglu and Nakajima (2013), McCall (1970), Morgan and Manning (1985), and Stigler (1961)). A useful survey of satisficing choice procedures can be found in Papi (2012).
Almost definitionally, models of incomplete preferences have to be concerned with satisficing: if the DM does not know his or her preferences, it is clearly impossible to find the best action. These models effectively impose satisficing as the only possible strategy. The problem here is that complete predictions of behaviour must also be impossible. Prediction is possible in models of behaviour under ambiguity. But here again the relevant information is available to the DM. U DM information is objectively correct, there is presumably always some action that is better than the one chosen by the DM. But here the DM does not choose to satisfice; nor does he or she choose how to satisfice. Models of rational inattention also capture t in that choice is made from a subset of the set of possible actions those which capture the attention of the DM, that is, those which are in the consideration set of the DM. However, these theories are silent on the reasons for the formation of a consideration set, and, in some of them, on how the consideration set is formed.
We examine a new theory that of Manski (2017)  where it is assumed that it can be immediately seen whether an alternative satisfies the aspiration level or T " and some differences. In some ways our test is closest to that of Hayashi and Wada (2010), though they test minimax, -maximin and the (linear) contraction model (Gadjos et al (2008)). We test

M
In the next section we describe the Manski model, while in section 3 we discuss the experimental design. Our results are in section 4, and section 5 concludes.

MANSKI S MODEL OF SATISFICING
In the model the DM has to choose some action. The DM knows that there is a set of actions, each member of the set implying some payoff. Crucial to the model is that the assumed objective of the DM is the minimisation of maximum regret (MMR). One reason for this is that there is no known probability distribution of the payoffs, so, for example Expected Utility theory and its various generalisations cannot be applied 2 .
Additionally, and crucially for our experiment, the solution is an ex ante solution, saying what the DM should plan to do as viewed from the beginning of the problem. As Manski I study ex choosing a strategy at the beginning of the problem, and then implementing it. This implies a resolute decision-maker. If the DM is not resolute the solution may not be applicable.

M
The maximin criterion gives the uninteresting result that the person should always choose the null option when deliberation is costly.
The paper applies the ex ante minimax-regret rule to this environment and derives a set of simple, yet intuitive, decision criteria for both the static and the dynamic choice situation. Simon (1955) also suggested that there can be a sequence of deliberations/satisficing where the DM adjusts his or her aspiration level in the light of information discovered. Hence, the dynamic choice situation is M P is that: (1) The optimal (maximum) number of rounds of deliberation (M*) if the DM uses a satisficing strategy is given by: (2) If the DM uses a satisficing strategy, the DM sets the aspiration level t m in the m satisficing as follows: The intuition of the theory is simple. Deliberation costs play a central role. O or Satisficing will be the decision if their respective associated cost (K,k) is low enough. If both costs are sufficiently large then N D will be preferred. If Satisficing is chosen, the aspiration level is midway between the relevant lower bound and the relevant upper bound, while the number of deliberation rounds is decreasing in its associated cost. This theory is different from the existing search literature in that it provides the concept of satisficing search that follows more closely " e aspiration levels than standard search models. It clearly states when the DM should satisfice. It also provides a solution to the choice of aspiration levels.
Before we move on to the experiment, let us briefly translate the above theory into a description of behaviour. The DM starts with knowing that there is a set of payoffs (the number of them unknown) lying between some lower bound L and some upper bound U. The DM is told the values of k and K. The first thing that the DM needs to do is to design a strategy. This depends on the values of k and K. If these are sufficiently large (see 3c above), the DM decides not to incur these costs and chooses N D . The DM is then told and given the payoff of the first action in the choice set, and that is the end of the story.
If K is sufficiently small (see 3a above) the DM decides to incur this cost O and hence learn the highest payoff. He or she gets paid the highest payoff minus K, and that is the end of the story.
The interesting case is 3b, where k is sufficiently small and K sufficiently large. The DM then decides to satisfice with (a maximum 3 of) M* rounds (as given by 1 above) 4 of satisficing. In each of these M* rounds, the DM sets an aspiration level, pays k, and is told at the end of the round whether or not there are payoffs greater than or equal to the stated aspiration level. More precisely, the DM is told whether there are 0, 1 or more than 1 payoffs greater than or equal to the stated aspiration level .The DM then updates his or her views about the lower and upper bounds on the payoffs in the light of the information received. This updating procedure is simple: This paper reports on an experiment to test the theory. We test whether subjects choose between N D , Satisficing and Optimising correctly (as in (3) above). We also test, when subjects choose to satisfice, whether they choose the correct number of rounds of satisficing (as in (1) above), and whether aspiration levels are chosen correctly (as in (2) above).

EXPERIMENTAL DESIGN
The actual experimental design differs in certain respects from the design of the theory. First, we told subjects that if they N D lowest payoff in the choice set, rather than the payoff of the first-ordered element of the choice set. Second, we only told subjects, when they chose to satisfice with an aspiration level t, whether there were or were not payoffs greater than or equal to t, and not whether there were 0, 1 or more than 1.
Moreover, if after satisficing for m rounds, and discovering that there were payoffs in a set [L m ,U m ], N D t point they would get a payoff equal to the lowest payoff in the set [L m ,U m ] minus mk. These differences do not change the predictions of the theory in that an MMR decision-maker will always assume that the first element is the lowest element.
Additionally, the ex ante choice of M* remains the same.
Let us give an example (which was included in the Instructions to the subjects). To make this example clear, we need to introduce some notation: the variable lvgeal is defined as the lowest payoff greater than or equal to the highest aspiration level for which there are payoffs greater than or equal to the aspiration level.
On the screen (see the screenshot below) there were three buttons The one on the left corresponds to N D , the one in the middle to " and the one on the right to O . In this example k=1 and K=10.
Suppose though the DM does not know this and our subjects were not told this that the payoffs are 55 18 75 19 9 If the DM clicks on the left-hand button straight away the income would be 9 (the lowest payoff).
If the DM clicks on the right-hand button straight away the income would be 65 (the highest payoff,

75, minus K).
If the DM clicked on the middle button and specified an aspiration level of 40, he or she would be told that there are payoffs greater than this, but would not be told how many nor what they are.
The software would, however, note that the lowest payoff greater than or equal to 40 is 55. This would be the lvgeal defined above. If the DM clicked on the left-hand button at this stage his or her income would be 54 (lvgeal minus k). A DM L 1 and U 1 are 40 and 100 respectively.
If the DM now clicks on the middle button again and now specifies an aspiration level of 70, he or she would be told that there are payoffs greater than this, but would not be told how many nor what they are. The software would, however, note that the lowest payoff greater than or equal to Subjects could keep on clicking on the middle button as often as they wanted, but they were told that the cost would be deducted from the payoff each time.
Note that in this particular case, it is better to click on the middle button twice (with aspiration levels of 40 and 70) and then on the left-hand button, rather than to click on either the left-hand button or the right-hand button straight away, and better than to click on the middle button one or three times (with aspiration levels of 40, 70 and 80) and then on the left-hand button. But this is not always the case.
In the experiment, 48 subjects were sequentially presented with 100 problems on the computer screen, all of the same type. They were given written Instructions and then shown a PowerPoint presentation of the instruction before going on to the main experiment. Subjects were informed of the lower (L) and upper (U) bounds on the payoffs in each problem; these were fixed at 1 and 100 respectively. They were also told the two types of cost; the cost of finding out whether there are any payoffs greater or equal to some specified aspiration level (k) and the cost of finding the highest payoff (K). The number of payoffs (N) was fixed at 5, though subjects were not given this information 5 . We used the procedure in Stecher et al (2011)    and professional degree (1). 46 subjects reported themselves as a student (8 subjects in a bachelor degree, 9 subjects in a master degree and 11 subjects in doctoral degree); one subject was a member of staff at the University of York; one subject did not report his/her current " re mainly White (26 subjects) while 18 were Asian/Pacific Islander, 3 were Black or African American and 1 other. There were only 5 subjects who had any work experience related to finance or economics, but most of them (34 subjects) had previously participated in an economics experiment.
To be a fair test of the theory, we need to give incentives to the subjects to act in accordance with it. We should repeat the fact that the theory is an ex ante theory: it tells DMs what to do as viewed from the beginning of a problem; it assumes commitment. Clearly, given the nature of the experiment, we cannot observe what the subjects plan ex ante, nor can we check whether they implement their plan. All we can observe is what they do, so we are testing the theory in its entirety meaning the validity of all its assumptions 7 . Ex ante the objective of the theory is to minimise the maximum regret. Ex ante Regret is the difference between the maximum possible income and their actual income. The maximum possible value of the former is exogenous it depends upon the problem which in our case is always 100 ex ante. So minimising the ex ante maximum regret is achieved by maximising their income. So we paid them their (average 8 ) income.
A payment from the experiment was their average income from all 100 problems plus the show-up fee of £2.50. Average income was expressed in Experimental Currency Units (ECU). Each ECU wa ECU s equivalent to £1. They filled in a brief questionnaire after completing all problems on the computer screen, were paid, signed a receipt and were free to go.
The average payment was £13.05. This experiment was run using purpose-written software written (mainly by Paolo Crosetto) in Python 2.7.

RESULTS AND ANALYSES
The purpose of the experiment was to test Proposition 2 of Manski (2017) as stated in section 3.
First, we compare the actual and theoretical decisions for all subjects and in each treatment.
Second, we compare the actual and theoretical predictions for income and regret. Third, we analyse the number of rounds of satisficing by comparing the theoretical and actual number for all 7 An alternative design would be to ask subjects to state a plan and then we B is not straightforward not only would subjects have to state whether they want to have N D O " A subjects to do this would be immeasurably more difficult than asking them to play out the problems. We expand on this in our conclusions. 8 If subjects are maximising their income on each problem they are maximising their average income, and vice versa, as problems are independent.
subjects and both treatments. Finally, we analyse t s and compare them with those of the theory.

When to Satisfice
Our experiment gives us 4,800 decisions (between N D , Satisficing and Optimising ) across 48 subjects and 100 problems.

4,800
The number in parentheses indicates the percentage by row and column In Table 2 we compare the actual and theoretical average income and average regret. Obviously, it must be the case that actual regret is higher than the theoretical regret (as subjects were not always following the theory). Subjects also have a higher average income. This suggests that subjects may have been working with a different objective function 10 , or making some assumption 9 Tables reporting results for treatment 1 and treatment 2 can be found in the Appendix. 10 For example, maximising Expected Utility. about the distribution of the payoffs that was not true 11 . Comparing the two treatments, we see that subjects in Treatment 2 have relatively better results in terms of the average income (33.40 ECU to 30.10 ECU) and regret (95.20 ECU to 121.10 ECU) than in Treatment 1. This is interesting, as the idea of Treatment 1 (where each problem was repeated 25 times) was to give subjects a chance to learn; we had expected performance to be better there. Perhaps they learnt about the refore departed from the theory?  Table 3 compares the theoretical (maximum 12 ) and the actual number of rounds of satisficing (obviously restricted to the cases where they actually satisficed). There are 452 problems out of 3,120 problems (14.49%), where the subjects should satisfice, and where they choose the same number of rounds of deliberation as the theoretical prediction. The difference between treatments is small: 16.67% and 11.89% matches of theoretical and actual number of rounds of deliberation, for treatments 1 and 2 respectively. Generally they choose fewer rounds of satisficing than the theory predicts 13 . 11 For example, assuming that the distribution was uniform. 12 Note that if subjects were following the theory with our design, the actual number of rounds would be equal to the M*, while in the theory the actual number could be less than M* (because they would stop satisficing if they discovered the highest payoff). 13 This is not a consequence of our experimental design which encourages subjects to choose the maximum number of rounds. Indeed with the theory we might observe numbers below the theoretical maximum.   We exclude the few outliers when the subjects put their aspiration level above 100. There were 39 (1.2%) out or out of 3347 cases where this happened.

How to Satisfice
We now investigate more closely whether subjects set their aspiration level as the theory predicts: equal to the mid-point between the relevant upper and lower bounds. We report below regressions of the actual aspiration level against the optimal level. If the theory holds, the intercept should be zero and the slope should be equal to 1. We omit observations where the aspiration level was above the upper bound (see footnote 9), and accordingly, carry out truncated regressions.
Before we proceed to the regressions, we note that the correlations between the actual and theoretical aspiration level 0.544 over all subjects, 0.513 for Treatment 1 and 0.569 for Treatment 2. Note: *indicates significance at 1% against the null that the true is 1.0 or 0.0 as appropriate. Table 4 shows that the coefficient on the theoretical aspiration level is not significantly different from 1 in Model 1. However in Model 1 we have included a constant term which should not be there; unfortunately it is significantly different from 0, which it should not be. If we remove the constant term to get Model 2, we find that the slope coefficient is almost significantly different from 1. So this table tells us that su M We broke down the analysis of Table 4 by treatments. The results are similar for Model 1 in both treatments. In Model 2, we find that the slope coefficient is significantly different from 1 in both treatments.
We now delve deeper and try to understand how the actual aspiration levels are determined, and in particular, how they are related to the upper and lower bounds. We present below regressions these bounds. If following the theory the relationship should be AL im = 0.5L im + 0.5U im (where AL im is subject i m of satisficing and L im and U im are the relevant lower and upper bounds). As before, we have excluded outliers (aspiration levels greater than the upper bound) from the regression and performed truncated regressions. Note: *indicates significance at 1% against the null that the true is 0.5 or 0.0 as appropriate. Table 5, over all the subjects, shows that the estimated parameters on the bounds are significantly different from the theoretical value of 0.5, and that the subjects put more weight on the upper bound and less on the lower bound when they select their aspiration levels.
If we break down the analysis of Table 5 by treatments, we see some differences between them. In Treatment 1 the estimated parameters are significantly different from the theoretical 0.5 (with more weight put on the upper bound than the lower), while in Treatment 2 they are much closer (and indeed only significantly different from 0.5 for one estimated parameter). So in Treatment 2 the subjects are closer to the theory in this respect than in Treatment 1. This confirms an earlier result. Possibly it was a consequence of the fact that in Treatment 2 each problem was an entirely new one, while in Treatment 1 (where 4 problems were given in blocks of 25) subjects were 15 and thus departing from the theory: as the subjects were working through the 25 problems they felt that they were getting some information about .

CONCLUSIONS
The overall conclusion must be that subjects were not following the part of the theory regarding the choice N D " O , possibly as a consequence of our experimental design 16 . However, the choice of the number of rounds of satisficing is closer to the theory. The first of these is a particularly difficult task and the second slightly less difficult, and therefore these results may not be surprising. In addition, subjects may have experienced difficulties in understanding what was meant by an ambiguous distribution.
However, when it comes to the choice of the aspiration levels, subjects are generally close to (though sometimes statistically significant from) the optimal choice of (L+U)/2. This latter task is easier and more intuitive. " heory is not empirically validated, while part of receives more empirical support. We tried to incentivise the use of the MMR preference functional by our payment rule, but the subjects could well have had a different objective function 17 . Unfortunately it seems difficult to force commitment on the subjects, and they may well have been revising their strategy as they were working through a problem. Nevertheless subjects seem to have been following the theory in at least one key respect the choice of their aspiration levels.

2,400
Note: the number in parentheses indicates the percentage by row.