Ambiguity aversion under maximum-likelihood updating

Maximum-likelihood updating (MLU) is a well-known approach for extending static ambiguity sensitive preferences to dynamic set-ups. This paper devel-ops an example in which MLU induces an ambiguity averse maxmin expected utility (MEU) decision-maker to (1) prefer a bet on an ambiguous over a risky urn and (2) be more willing to bet on the ambiguous urn compared to an (ambiguity neutral) subjective expected utility (SEU) decision-maker. This is challenging, since prior to observing (symmetric) draws from the urns, the MEU decision-maker (in line with the usual notion of ambiguity aversion) actually preferred the risky over the ambiguous bet and was less willing to bet on the ambiguous urn than the SEU decision-maker. The identiﬁed switch in betting preferences is not due to a violation of dynamic consistency or consequentialism. Rather, it results from MLU’s selection of extreme priors, causing a violation of the stability of set inclusion over the course of the updating process.

It is crucial that the switch in betting preferences identified in this paper is rooted neither in dynamic inconsistency nor a violation of consequentialism. To clarify this, the paper adopts the framework of Epstein and Schneider (2007) which respects dynamic consistency as well as consequentialism. 1 The other reason to follow Epstein and Schneider (2007) is their explicit use of MLU. Concrete, they adopt the generalized and less extreme MLU, already suggested by Gilboa and Schmeidler (1993), in which also priors that only "epsilon maximise the likelihood function" are updated. This paper demonstrates that MLU, both in the strict and the generalized form, gives rise to the switch in betting preferences surrounding risky and ambiguous urns. The deeper reason is that MLU does not respect set inclusion stability over the course of the updating process.
The paper proceeds as follows. Section 2 presents the simple example in which exante and ex-post betting preferences are surprisingly unaligned. Section 3 presents the underlying framework of learning under ambiguity, a simplified version of Epstein and Schneider (2007). Thus equipped, Sect. 4 will revisit the example to understand the deeper reason for the switch in betting preferences. Section 5 concludes.

Guiding example 2.1 Betting on urns with unknown composition
The example presented in this section revolves around the two urns depicted in Fig. 1. Both urns contain exactly three balls. Apart from a white and a black ball, each urn contains a third ball that is either black or white. The composition of either urn, which does not change over the course of the experiment, is unknown to the decisionmaker. The key difference between both urns is that the decision-maker knows that the colour of the third ball in the 'risky' urn R has been determined via an equiprobable mechanism, e.g., a fair coin. For the ambiguous urn A, however, there is no information about the mechanism that determined the colour of the third ball.
In every period and for each urn, one ball is drawn and then put back (sampling with replacement). The repeated observations enable the decision-maker to learn about the composition of the urns. The objects of choice are simple binary bets of the form x Urn Colour y on the colour of the next ball drawn. The bet 1 A B 0, for instance, is based on the colour of the ball drawn from urn A and involves a payment of 1 $ if the ball is black, and 0 $ otherwise. 2

Two decision rules
The decision criterion for evaluating bets on urn R is quite straightforward. The standard Bayesian approach is to start with a prior belief that assigns equal probability to both composition scenarios and then repeatedly update this prior belief upon observing more and more draws. Bets are ranked according to expected payoffs based on the current belief. 3 For urn A, the uniform prior distribution lacks the clear 'objective' justification. We compare decision-makers following two prominent approaches in light of ambiguity. The first decision-maker maximises subjective expected utility (SEU, Savage 1954), which involves the formation of a unique (subjective) initial prior (in the absence of further information typically the uniform distribution) about the composition of urn A that is used to rank bets according to the associated expected payoffs. The natural dynamic extension of SEU is standard Bayesian updating. The second decision-maker maximises maxmin expected utility (MEU, Gilboa and Schmeidler 1989). In contrast to SEU, MEU preferences involve a set of beliefs, and bets are ranked according to minimal expected payoffs. The dynamic extension of MEU preferences employed here is MLU as axiomatized by Gilboa and Schmeidler (1993) and used in Epstein and Schneider (2007). The updating process will be explained in detail in Sect. 3. We compare betting preferences along two dimensions. First, for each decisionmaker separately, we will compare whether the decision-maker prefers a bet on the risky or the ambiguous urn. Second, for each urn separately, we will compare the willingness to bet (expressed as certainty equivalents) of both decisionmakers.

Surprising choices
Assume that the decision-makers are currently characterized by the following belief structure. For urn R, both decision-makers assign the probability 2/3 to the scenario that the unknown ball is black. Let this also to be the SEU decision-maker's assessment of urn A, for instance, because she held the same initial prior and observed the same history of draws. In contrast, the MEU decision-maker's belief structure regarding urn A is assumed to be quite different; her set of beliefs has a tendency towards the two-black-one-white scenario in the sense that this scenario's probability ranges from 3/4 to 1. In particular, this set of beliefs does not contain the 'weaker' 2/3, currently held by the SEU decision-maker.
It is a simple task to determine the preferences over the standard bets on a black ball, 1 R B 0 and 1 A B 0, that result from this belief configuration. Since her beliefs coincide, the SEU decision-maker is indifferent between betting on the risky and the ambiguous urn. In contrast, the MEU decision-maker strictly prefers a bet on the ambiguous urn A over the same bet on urn R. For urn R, as their beliefs coincide, the SEU and MEU decision-makers have the same willingness to bet on the next ball being black, expressed as a same certainty equivalent. For urn A, however, the MEU decision-maker is more willing to bet than her SEU counterpart. This is unusual. Ambiguity aversion in general and MEU in particular are typically associated with a preference for risky rather than ambiguous situations. However, to be clear, neither the MEU decision-maker's preference for the ambiguous over the risky urn nor the higher certainty equivalent (as compared to the SEU decision-maker's) stands in internal conflict with the axioms of ambiguity aversion. Actually, the MEU decision-maker is ambiguity averse given her beliefs: she evaluates the bet 1 A B 0 based on her worst belief, here 3/4. The reason for the untypical betting preferences is that this worst belief is still more 'optimistic' than her belief about urn R and the SEU decision-maker's assessment of urn A. In summary, we should not regard the MEU decision-maker's inclination to bets on the ambiguous urn as problematic per se; it may simply be that she received more conclusive evidence for the scenario that the unknown ball in urn A is black.
What, however, actually makes this example surprising and noteworthy is that this is not the case. Rather, the information both decision-makers received about both urns was fully symmetric. Initially, the level of information was symmetric in the sense that both decision-makers had not observed any draw from either urn. And, indeed, at this ex-ante stage-and in accordance with usual notions of ambiguity averse behaviourthe MEU decision-maker strictly preferred a bet on urn R over urn A and was less willing to bet on urn A than the SEU decision-maker. Then, again preserving symmetry across agents and urns, both decision-makers observed the draw of one black ball from either urn. This information led to the ex-post configuration of beliefs and associated unusual betting preferences described above. Table 1 summarises the example.
To understand why this symmetric information had the potential to induce such unintuitive switch in betting preferences, it is needed to have a closer look at the framework behind the example and in particular the maximum-likelihood updating process. This is the focus of the subsequent Sect. 3. Thus equipped, we can revisit the example in Sect. 4. Table 1 Summary of the switch in betting preferences between before and after observing the draw of black balls from either urn

Stage
Comparing urns Comparing decision-makers The comparison of certainty equivalents is based on the standard bets on a black ball, 1 R B 0 and 1 A B 0

Dynamic MEU preferences with maximum-likelihood updating
The framework to verify the switch in betting preferences identified in the previous section is a simplified version of Epstein and Schneider (2007). In particular, the two urns presented above are reduced versions of the scenarios in Epstein and Schneider (2007).

Fundamentals
The ratio of black balls in the urn is the unknown parameter θ with possible values in the parameter space identical for all times. We denote by s t ∈ S the colour observed by the agent at time t. An agent's information at time t is the history s t = (s 1 , . . . , s t ). The natural full state space is S ∞ . The evaluation criterion for choices about bets on urn R is straightforward. Starting with the uniform prior μ 0 = (1/2, 1/2) over the parameter space = {1/3, 2/3}, the decision-maker uses Bayesian updating to process the information provided by the observed signal history s t to a posterior μ t . This posterior, in turn, is directly associated with a 'one-step ahead belief' p t (·) = l(·|θ)dμ t (θ ) on S about the colour of the next ball. Here, l(·|θ) is the likelihood of a draw s ∈ S given a composition of the urn θ , i.e., l(s = B|θ) = θ and l(s = W |θ) = 1 − θ . To rank different bets, the decision-maker compares expected payoffs. For instance, the expected payoff π of bet x R B y given the one-step ahead belief p t (·) is There are different decision criteria for bets on urn A. The SEU decision-maker forms a unique prior about the composition of the urn and then follows the standard Bayesian updating procedure, so that the decision rule coincides with that in (1). In the absence of any observations, the uniform distribution is a plausible prior and the betting behaviour regarding urn R and urn A indistinguishable. In contrast to SEU, the intertemporal MEU decision-maker initially holds a set of priors M 0 that is, upon arrival of new information, updated to a set of posteriors M t . This updating process is characterized by MLU; the next subsection explains this process in detail. The set of posteriors M t is, in turn, associated with a set of 'one-step ahead beliefs' S about the colour of the next ball. The MEU decision-maker ranks bets according to maxmin expected utility. Given P t (s t ), the maxmin expected payoff of bet x A B y is Note that the model presented here is much simpler than the framework of Epstein and Schneider (2007). Here, the decision task is structurally the same at all points in time. In particular, past choices in no way restrict the remaining decision-tree.
In addition, the current choices do not impact the 'informativeness' of the next ball drawn; there is no room for experimentation and similar considerations of giving up short-term payoffs for long-term information benefits (see, for instance, Moscarini and Smith 2001). Every decision just focuses on the colour of the next ball and has to be made solely based on what the decision-maker has learned about the composition of the urn. In particular, discounting does not play a role and can be ignored.

Maximum-likelihood updating
This subsection describes the process of updating the set of priors M 0 to the set of posteriors M t by MLU. It proves useful to define the plausibility of a prior μ 0 ∈ M 0 given the data s t = (s 1 , . . . , s t ) by Plaus(μ 0 ; s t ) = t j=1 l(s j |θ)dμ 0 (θ ). ( We use the generalization of MLU presented in Epstein and Schneider (2007) and already suggested in the classical paper (Gilboa and Schmeidler 1993). Generalized MLU with parameter α ∈ (0, 1] only updates priors with α-maximal plausibility given the signal history s t . Concrete, the set of posteriors M t (s t ) is the priorwise update of the set of admissible priors: The strict MLU is the case α = 1 in which only priors with maximal plausibility are updated. In contrast, full Bayesian updating is the corner case α = 0-explicitly not included in the present paper and in Epstein and Schneider (2007)-in which all priors are updated unconditionally. The example presented in Sect. 2 and revisited in Sect. 4 is characterized by α = 4/5.

Ex-ante choices
The example presented in Sect. 2 is based on the following plausible assumptions about the ex-ante belief structure. For urn R, it is natural that initially both decision-makers assign equal weights to the two scenarios θ = 1/3 and θ = 2/3. Formally, they hold the initial prior μ 0 = (1/2, 1/2) over the parameter space = {1/3, 2/3}. In line with the principle of insufficient reason, this is also the prior that the SEU decisionmaker holds regarding urn A before receiving any information about its composition. In contrast, the MEU decision-maker is assumed to initially hold the full prior set This belief configuration directly leads to the following ex-ante betting preferences. The SEU decision-maker is, irrespective of the bet, indifferent between betting on urn R and urn A as p 0 (s = B) = 1/2 · 1/3 + 1/2 · 2/3 = 1/2 in any case. As a consequence, the certainty equivalent of the standard bet on black is CE SEU (1 R B 0) = CE SEU (1 A B 0) = 1/2. In contrast, the MEU decision-maker bases here decisions on the worst prior in M 0 . For a bet on black, this is the prior (1, 0) that puts maximal weight on the scenario with only one black ball in the urn, θ = 1/3. Accordingly, the relevant p 0 ∈ (S) for evaluating expected payoffs is 1 · 1/3 + 0 · 2/3 = 1/3, so that CE MEU (1 A B 0) = 1/3, while CE MEU (1 R B 0) = 1/2. In the decision set-up before observing the first draw from the urns, the MEU decision-maker strictly prefers to bet on the risky urn R, in particular 1 A B 0 ≺ M EU 1 R B 0, and her certainty equivalent for bets on urn A is smaller than SEU's, CE MEU Table 1). This is in line with the usual notion of ambiguity aversion.

Ex-post choices
We now turn to the situation in which the decision-makers have observed the draw of a black ball from both urns. For urn R, on which both decision-makers agree, standard Bayesian updating transforms the prior μ 0 = (1/2, 1/2) to μ 1 = (1/3, 2/3), reflecting the increased subjective probability for the scenario that the unknown ball is black. This is also how the SEU decision-maker interprets the black draw from urn A. As a result, the SEU decision-maker is still indifferent between betting on urn R and urn A; in addition, the certainty equivalent of a bet on urn R is assessed equally by both decision-makers (cf. Table 1). The MEU decision-maker naturally has a different take on Urn A. To recapitulate the procedure explained in Sect. 3.2, the first task is to find the most plausible theory μ 0 ∈ M 0 . This is (0, 1). The plausibility of this theory, cf. (3), is 2/3. With α = 4/5, the MEU decision-maker rejects all theories with a plausibility less than 4/5 · 2/3 and thus keeps the set M 0 (s) = {(ν, 1 − ν) | 0 ≤ ν ≤ 2/5}. Finally, this set is  B 0). This is a switch to the ex-ante configuration and noteworthy insofar as the information that caused this switch was symmetric across urns and decision-makers.
Another way to illustrate the surprising behaviour is to compare two different MEU decision-makers. Besides the MEU decision-maker starting with the full prior

The cause of the switch
This raises the question about the origin of the switch just presented, both in the comparison of SEU and MEU as well as MEU and MEU. 5 One explanation might be that the unintuitive betting preferences after updating occur, because the intertemporal formalization of MEU fails to ensure ambiguity aversion. However, this is not the case. Given her beliefs, the MEU decision-maker is, in fact, ambiguity averse. Through the min-operator in (2), she is ambiguity averse by construction.
Another possible narrative for the switch is dynamic inconsistency. One may tend to think-even more so as dynamic inconsistency is a typical deviation from standard rationality assumptions with ambiguity averse preferences-that the decision-maker initially prefers bets on the risky urn, but cannot sustain, for whatever reason, this preference over time. Yet, this narrative does not apply here. The decision-makers actually do not make any plans for future choices and only have preferences over the very next bet. And even if their current choices depended on future plans, dynamic consistency would be ensured, because it is written into the recursively defined framework of Epstein and Schneider (2007).
The actual reason for the switch in betting preferences is the belief dynamics under MLU, in particular MLU's focus on 'extreme' priors. This gets most obvious in the comparison of the two maxmin decision-maker MEU and MEU in the last subsection: what the 'more pessimistic' decision-maker MEU with the full prior set identifies as the most likely prior after observing one black ball is certainty that the urn consists of one white and two black balls. For making the point stark, we can focus on strict MLU, α = 1, and see that this 'certainty' is in fact the only posterior the MEU decision-maker bases her decision on. In contrast, the 'less pessimistic' decision-maker MEU with a smaller prior set cannot regard certainty regarding the composition of the urn as the most likely prior, simply because this extreme case was not a prior to begin with. In summary, the property MLU violates is set inclusion stability of the set of beliefs over the learning process: Whenever the implication in (6) is violated, there is room for a switch in betting behaviour of some sort to occur. Full Bayesian updating, i.e., the corner case α = 0 in the above setting, however, trivially respects (6).

The general case
One may argue that the switch in betting preferences can be avoided by adequately choosing the rejection parameter α. In fact, any α < 3/4 would, at least at t = 1, prevent the example's switch in preferences. It thus seems in principle possible to restrict α to innocent values. The Appendix, however, demonstrates that this is not the case. For every α > 0, there is a setting similar to that considered in the example for which such a disconnect in the behaviour of the MEU decision-maker occurs. The proof involves a generalization of urn R and urn A with more than three balls and constructs a signal history that gives rise to the rejection of the uniform distribution, SEU's initial prior, and thus a violation of (6). See the Appendix for details. Overall, this demonstrates that MLU will always be prone to some switch in betting preferences.

Concluding discussion
Ambiguity sensitive preferences play-in theory and applications as well as for descriptive and prescriptive reasons-an increasing role in economics. Unlike subjective expected utility (SEU), there is no canonical way to extend static ambiguity sensitive preferences to dynamic settings. The two main competing approaches for this extension are full Bayesian updating and maximum-likelihood updating (MLU). This paper has made a contribution to the debate about these updating rules by demonstrating that the latter involves unintuitive ramifications. MLU induces an ambiguity averse maxmin expected utility (MEU) decision-maker to prefer bets on urns with unknown over urns with known composition and to have a higher inclination to bet on the ambiguous urn than her SEU counterpart, despite the absence of any asymmetries in information across urns and agents that might have caused this switch.
The paper has further demonstrated that neither a violation of dynamic consistency nor that of consequentialism-typical sources for deviations from standard rationality when extending ambiguity averse preferences to dynamic settings-is the source of this surprising switch. Rather, the paper has uncovered that MLU favours 'extreme' priors and hence does not preserve set inclusion stability of prior and posterior sets. Moreover, it has demonstrated that adopting a less strict MLU is not able to preclude the switch in betting preferences, irrespective of how moderate the rejection criterion is.
The choice to base the example on the framework of Epstein and Schneider (2007) was motivated by convenience and to ensure dynamic consistency right from the start. However, the findings of this paper extend to a broader set of models. It immediately extends to uncertainty averse capacities, as these can be expressed with multiple priors. However, the problematic switch in betting preferences is not restricted to ambiguity averse preferences and may thus occur both in alternative multiple prior models and for general capacities. An ambiguity loving max-max decision-maker, for example, can be expected to be surprisingly reluctant to bet on black upon having observed the draw of one white ball. The upshot here is that MLU has broad repercussions through its effect on the belief dynamics.
The findings of this paper are relevant for various reasons. In a normative dimension, it is hard to argue why a decision-maker that initially was more cautious in her description of the probabilistic environment, and accordingly less willing to expose herself to ambiguity, should change this and, upon observing information, become more willing to bet on this ambiguous source of uncertainty, both relative to a risky counterpart and relative to an SEU decision-maker. Accordingly, all economic applications that intend to model intertemporal ambiguity averse preferences ought to be aware of the unintuitive implications which MLU may have on their results. The obvious alternative is to rely on full Bayesian updating. To ensure learning with the set of posteriors converging to the true parameter, it is just needed to avoid that the initial prior set contains 'certainty' priors that are immune to re-evaluation. In a positive dimension, the findings of this paper raise the empirical question whether choices of decisionmakers are compatible or in conflict with the stability of set inclusion of beliefs. In a similar direction, recent experimental evidence shows that some individuals prefer ambiguous over risky bets, especially for unlikely events and losses (Trautmann and Kuilen 2015). This is interpreted as ambiguity seeking. The present paper shows that such ambiguity seeking behaviour can be compatible with ambiguity aversion given specific beliefs, hence adding a further dimension to the recent debate on ambiguity attitudes, in particular in dynamic settings.
following the principle of insufficient reason is the special case of MEU with M 0 being the singleton containing just the uniform distribution.
To show that no rejection parameter 0 < α ≤ 1 can prevent a switch in betting preferences, the strategy in this appendix is to prove that for any α, there is a pair of urns similar to urn R and urn A and a signal history s t for which M t ⊂ M t , hence violating (6). For this purpose, we start by generalizing the setting from urns with three balls to arbitrary symmetric settings. Then, we construct, for given rejection parameter α, a suited urn and signal, such that M t ⊂ M t .
The generalized urn R and urn A, denoted by U R (n, k) and U A (n, k), respectively, have exactly 2n + k balls. It is known that n balls are black, n balls are white, and each of the remaining k balls can either be black or white. As will become clear in what follows, the urns in the example in the main text correspond to the case n = k = 1. To generalize the three balls in urn example, we assume that the number of black balls within the k unknown balls in urn U R (n, k) is uniformly distributed. 6 The urn U A (n, k) is basically the same, but without information about the distribution of the k unknown balls. In either case, the parameter set is = {n/(2n + k), . . . , (n + k)/(2n + k)} with θ ∈ being the true fraction of black balls in the urn, unknown to the decisionmaker. The period state space is again S with the full state space S ∞ . The likelihood functions are fully specified by l(s = B|θ) = θ .
As mentioned above, let M 0 = ( ) be the full prior set. For M 0 , it is convenient to consider symmetric sets around the uniform distribution: (k) = (ν 0 , . . . , ν k ) | i ν i = 1 , 0 ≤ ≤ ν i ≤ 1 − k ≤ 1 ∀i , < 1 k+1 . (7) By construction, all these sets contain the uniform distribution. The full set of priors ( ) is the special case (k) with = 0, and for → 1/(k + 1), the set collapses to a singleton with the uniform distribution as the only element. Clearly, (k) ⊂ (k) if > . We now have the toolkit at hand to formulate and prove the main result.