Replicator Dynamics of Symmetric Ultimatum Game

We construct a symmetric version of the ultimatum mini game and analyze the stability of its equilibria in the replicator dynamics. We show that the mere symmetry does not lead to a unique social norm consistent with the observed experimental results. Moreover, the average offer in the population has to be lower than 50 % in order to have one of the equilibria consistent with the experimental data.


Introduction
Ultimatum game is the simplest bargaining game. Its goal is to divide a unit of a certain good between two players. The first player offers a fraction δ ∈ [0, 1] of the good to the second player. The second player can either accept the offer or reject it. In the first case, the second player receives the payoff δ and the first player 1 − δ, while in the second case both players receive nothing. The first player is referred to as "proposer" and the second player as "responder".
It is not difficult to see that in the ultimatum game any offer δ can be supported by a Nash equilibrium. Suppose a proposer offers a share δ > 0 and a responder accepts any offer not lower than δ. Any such pair constitutes a Nash equilibrium and consequently there is a continuum of Nash equilibria in this game. However, if a proposer deviates from this equilibrium and offers any positive share δ < δ, the best-reply strategy of a responder is to accept it. This shows that any such equilibrium is not subgame perfect. In fact, if the good is infinitely divisible there is the unique subgame-perfect equilibrium in which a proposer offers δ = 0 and a responder accepts any offer. 1 Bargaining games were extensively tested in experiments. It seems that the first experiment involving the ultimatum game was conducted in [5]. The average observed offer was between 31 % and 35 % and so it was nowhere near the subgame-perfect equilibrium. This experiment was replicated in [6] and in a larger design in [2,3,7,14]. A detailed discussion of these and other results is beyond the scope of this paper, a detailed survey can be found in [11].
The notion of the Nash equilibrium does not offer any prediction for the ultimatum game. On the other hand, the observed behavior of subjects playing the ultimatum game is different from the prediction of the subgame-perfect Nash equilibrium. Therefore, many models have been proposed to explain results of experiments. One strain of such models is learning approaches, cf. [1,7,8,12]. Specifically, in [4] the replicator dynamics has been used to model behavior of players in the ultimatum game. The main insight from [4] is that in the asymmetric ultimatum game, i.e. with a population of proposers and a population of responders, there are two asymptotically stable equilibria provided that both proposers and responders make errors. One equilibrium is near the subgame-perfect equilibrium but at the other one the average offer is about 20 %. It is speculated that due to a superficial similarity of the ultimatum game to real bargaining situations, a particular social norm is triggered that leads to initial conditions in the basin of attraction of the equilibrium supporting the experimental results: (. . .) we suggest that initial play reflects decision rules that have evolved in real-life bargaining situations that are superficially similar to the Ultimatum Game. These bargaining games generally feature more symmetric allocations of bargaining power than the Ultimatum Game, yielding initial play in Ultimatum Game experiments that need not be close to the subgame-perfect equilibrium.
Here we construct a symmetric model of the ultimatum bargaining game and use replicator dynamics to test whether the symmetry of the roles leads to any significant changes as compared to the original model in [4].
The paper is organized as follows. In Sect. 2.1, the basic model is introduced. In Sect. 2.2, the unperturbed version of replicator dynamics is analyzed while the perturbed version is addressed in Sect. 2.3. We discuss our results in Sect. 3 and conclude in Sect. 4.

Model
We adopt the standard evolutionary setting and construct a model with a single population of players randomly matched to play the mini ultimatum game. Since the mini ultimatum game is an asymmetric one, the roles in the game, i.e. proposer and responder, are assigned at random. Consequently, the position of all players ex-ante is exactly the same. We assume, though, the conditioning of behavior on the assigned role, that is, a strategy is a plan describing a behavior in both roles. We are interested in studying the evolution of a distribution of pure strategies (describing behavior in both roles).

Strategies and Payoffs
In a symmetric version of the ultimatum game, strategies of players must prescribe behavior in both proposer and responder roles. Therefore, a strategy of a player is a pair (α, β), where α ∈ [0, 1] is an offer of a player in the role of the proposer and β ∈ [0, 1] is an acceptance level, that is, a player in the role of the responder accepts only offers not lower than β. For simplicity we narrow down the possible offers to just three, α, ∈ {0, δ, 1}, where 0 < δ < 1. We also assume that acceptance levels cannot be larger than offers, that is, β ≤ α. Consequently, we are left with six possible strategies: The payoff matrices, one for each role (bold zeros correspond to rejections), read for a proposer and a responder, respectively, where the first strategy is the egoistic one, then follow two medium strategies, and finally three altruistic ones.

Unperturbed Replicator Dynamics
We consider a single population of players. They are repeatedly matched into pairs in which their roles are attached at random. Fractions of the population using given strategies are denoted by x 1 , . . . , x 6 , respectively. The vector of fractions is denoted by x ∈ , where is a simplex, Since we consider only a single population in which roles are attached randomly to players, average payoffs are given by the matrix A = (P + R T )/2, where R T is the transpose of R. For this matrix we consider the standard replicator dynamics of the forṁ where time derivatives are denoted byẋ i .
We first notice that all altruistic strategies disappear in the long run for any interior initial condition. 2 Namely, let ξ(x 0 , t) be the solution of (1) with the initial condition x 0 ∈ int( ), then lim t→∞ ξ i x 0 , t = 0 for i = 4, 5, 6. Therefore, we can consider only first three strategies. Such a game is reminiscent of the ultimatum mini game considered in [4] with the additional strategy (δ, 0). The replicator dynamics for variables x i , i = 1, 2, 3 takes the following form: We (2) can be written in the following form:ẋ The next three propositions follow directly from (3).

Perturbed Replicator Dynamics
We would like to see how equilibria and their stabilities change under perturbations of the replicator dynamics. Following [4] we consider the situation where the original evolution 2 Consider η ∈ (0, 1). For any x such that takes place with a probability 1 − and with a small probability strategies are chosen randomly, that is, each one with the probability 1/3. The random choice of strategies introduces the term (1/3 − x i ) into the replicator dynamics and we obtain the following perturbed version of the dynamics (2): Let us notice that from the last two equations of (4) it follows that the segment x 2 = x 3 is invariant under the flow (4). Moreover, x 2 = x 3 at any equilibrium. Hence we may have at most three equilibria of (4).
As we have already mentioned, a line x 2 = x 3 is invariant under the perturbed dynamics. This plays a crucial role in the following proposition.
Proposition 5 (Global Convergence) Let x ∈ be any initial condition and let ξ(x, t) be the solution (4). Then ξ(x, t) converges to one of the equilibriax i .
Proof System (4) is a planar system and so by the Poincaré-Bendixson theorem we know that ξ(x, t) converges either to a stationary point or a limit cycle. Therefore, in order to prove the proposition it is enough to show that there cannot be any limit cycles.
Suppose that ξ(x, t) converges to a limit cycle defining a region γ that is invariant under f(x, ). It is well known 3 that there must be a stationary pointx of f(x, ) such thatx ∈ int(γ ). But then a limit cycle has to cross the invariant line x 2 = x 3 which cannot happen.
To complete the analysis of the perturbed system we focus on local phase portraits around equilibria of (4). Proposition 6 (Local Phase Portraits) Let δ < 1/2 and > 0 be small enough. Equilibriâ x 1 andx 3 are asymptotically stable and equilibriumx 2 is a saddle.
The case of δ ≥ 1/2 is trivial. The proof of Proposition 6 works for the only equilibrium x 1 in the case of δ ≥ 1/2 as well. We could also argue differently. We know by Proposition 5 that any trajectory converges to one of the equilibria and all of them are on a line x 2 = x 3 . Therefore, trajectories come arbitrarily close to this line and by the continuity of (4) their behavior is qualitatively identical to the behavior of (4) on that line.

Discussion
The asymmetric noisy model introduced in [4] predicts two stable equilibria. The first one is close to the subgame-perfect equilibrium of the ultimatum game while the other one is substantially different supporting the average offer of around 20 %. It was speculated there that at the initial phases of an experiment, the observed behavior is driven by a social norm that happened to be triggered by the framing of the ultimatum game. This social norm is formed in superficially similar real-life bargaining problems where players are in a symmetric situation. In other words, the initial conditions of the replicator dynamics are likely to belong to the basin of attraction of the non-perfect equilibrium because the triggered behavior is formed in a (more) symmetric model.
In order to check this possibility, we constructed a symmetric model of the ultimatum game with proposer and responder roles assigned at random hence the situation of players is completely symmetric. So, how does our symmetric model compare? In the unperturbed case, the behavior of the model is simple. In fact, we can justify a population using a modal offer δ through a stable (but not asymptotically stable) equilibrium. However, we are not satisfied with such an explanation for two reasons. Firstly, we would like to have an asymptotically stable equilibrium supporting the observed behavior, and secondly, in the ultra-long run one would expect some kind of a noise in the system. This leads us as in [4] to consider a perturbed version of the replicator dynamics.
Behavior of the perturbed replicator dynamics is to a certain extent similar to the behavior of perturbed dynamics in the original ultimatum bargaining mini game. For δ < 1/2 there are three equilibria. Two asymptotically stable equilibria are connected through an unstable manifold of the saddle point. Equilibriumx 1 is the perturbed subgame-perfect Nash equilibrium and cannot be used to explain the behavior observed in experiments. There is, however, the second asymptotically stable equilibriumx 3 , at which most of the population uses the social norm offer δ. This equilibrium can explain to a certain extent the behavior of subjects playing the bargaining game in a lab.
Although, we can use the equilibriumx 3 , there are some problems with this explanation. Firstly, the experimental results on bargaining games vary wildly and in the context of this work the only common observation is that the average offer is lower than 1/2. Thus to calibrate our model for various data we would have to use different noise levels . This is a minor problem since we can always argue that in a lab only a very small sample is observed and this can vary even though there is a common noise level. 4 There is a more serious problem with the use of the equilibriumx 3 as the explanation of the observed behavior. Although this equilibrium is asymptotically stable, it is not globally asymptotically stable. If an initial population state belongs to the basin of attraction ofx 3 and there are only small uncorrelated jumps, then in the ultra-long time, the population state should be near the equilibriumx 3 . However, if there are large perturbations or a series of correlated small perturbations, the population state may very well happen to be in the basin of attraction of the subgame-perfect Nash equilibriumx 1 . If the perturbations were persistent, then we could observe a system jumping between equilibria which is reminiscent of [15].
One way of estimating how likely it is for a population state to be in a given basin is to look at the basins area. For precise calculations we would need to have formulas for the stable manifold of the saddle pointx 2 and these are notoriously difficult to get. However, for δ = 1/3, the segment x 2 = δ is the stable separatrix of the saddle pointx 2 and consequently the ratio of the area of the basin of attraction ofx 3 to the whole simplex is 4/9. Numerical calculations suggest that for δ < 1/3 this separatrix is to the left of the segment x 2 = δ and for δ > 1/3 it is to the right of this segment. As δ ↓ 0, the area of the basin of attraction of the subgame-perfect equilibriumx 1 decreases to 0, but at the same time the difference between offers of the egoistic strategy and the medium strategies decreases as well. If the plausibility of a given equilibrium is measured by the size of its basin of attraction, then there is a trade off between the plausibility ofx 3 and how different medium strategies are from the egoistic one.
Before we continue with final conclusions, we would like to compare closer our game with the original one considered in [4]. The original ultimatum mini game in strategic form reads (δ, δ) (0, 0) reject accept (δ, δ) high (2, 2) (2, 2) (0, 0) low (0, 0) (3, 1) where we overlaid our corresponding strategies. We deal with a slightly different game, where the payoffs read (δ, δ) (0, 0) reject accept (δ, δ) high (2, 2) (2, 2) (0, 0) low (0, 0) (4,0) where in both matrices we use the original payoffs, i.e. players divide the pie of size 4. The obvious difference between above two games concerns the "low offer" strategy. In the original paper, the offer is low but still positive making rejection costly. In the game considered herein, the low offer is just plain nothing thus the responder is indifferent between rejecting and accepting the offer. It is a valid question if such a change influences our results. To answer this question we need to introduce the minimal offer which we denote by μ > 0, i.e. α, β ∈ {μ, δ, 1}, where 0 < μ < δ < 1. The introduction of the minimal offer μ does not change the first step in our analysis, the last three strategies vanish in the long run and so we can consider the 3 × 3 game with the first three pure strategies.
Again, as in the previous analysis, we first consider a scenario without noise. The behavior of such dynamics is depicted in Fig. 3(a). The phase portrait is qualitatively similar to the phase portrait of the previously considered dynamics depicted in Fig. 1(a). The set of equilibria E is a union of four components with identical stability properties. The only difference occurs with respect to the component E 2 which is reduced to just a single point E 2 = (1 − δ, δ, 0) and is unstable. This phase portrait is extremely similar to the Fig. 2 in [4], p 76. In this case, an introduction of symmetry does not lead to any substantial change regardless of the value of the minimal offer μ.
The next step is the analysis of the perturbed dynamics. Figure 3(b) depicts the phase portrait of the perturbed dynamics with the positive minimal offer μ > 0. Qualitatively the situation is exactly the same as the one with a noise and μ = 0 depicted in Fig. 1(b). There are three equilibria. The first onex 1 is close to the first pure strategy corresponding to the subgame perfect equilibrium of the original game and is asymptotically stable. The third onẽ x 3 is also asymptotically stable equilibrium and corresponds to thex 3 equilibrium of the dynamics (4). There is an unstable equilibrium (saddle)x 2 corresponding to thex 2 equilibrium of the dynamics (4). Also in this case the introduction of the positive minimal offer μ > 0 does not change anything qualitatively.
Introduction of the minimal positive offer μ > 0 neither influences results qualitatively nor gives an additional insight. On the other hand, the assumption that μ = 0 simplifies immensely calculations and this is the main reason why we decided to analyze in detail this version of the game. Now we ask ourselves: is there anything that we can conclude from the replicator dynamics of the symmetric ultimatum mini game? The only good news is that if we would like to have any chance to explain the experimental data, we have to have δ < 1/2, a result that goes quite well with the collected data. However, the symmetry does not lead to the creation of the social norm, triggered by the framing of the ultimatum game during experiments, different from the subgame-perfect equilibrium of the ultimatum game. Regardless of the symmetry, there are still two asymptotically stable equilibria and one of them is arbitrarily close to the subgame-perfect equilibrium of the ultimatum game. Consequently, the symmetry argument cannot be used to justify the appropriate initial conditions in the replicator dynamics for the asymmetric ultimatum mini game.

Conclusions
We proposed an extended symmetric version of the ultimatum mini game introduced in [4]. We analyzed the global behavior of the model in both unperturbed and perturbed cases. The main insight from the analysis presented in this paper is that the mere symmetry cannot be solely responsible for a creation of the social norm supporting an equilibrium that is not subgame perfect. The main problem of the model introduced in [4] is still unsolved. It seems to us that to fully explain results of experiments one has to construct dynamics with a unique globally asymptotically stable equilibrium consistent with the observed experimental data. Also, it seems to us that any dynamics based on imitation, like the replicator dynamics, is not a suitable model of learning in the context of bargaining situations. Construction of a model based on different learning procedures with a unique globally asymptotically stable equilibrium is left for a future research.