A GameTheoretical Winner and Loser Model of Dominance Hierarchy Formation
Abstract
Many animals spend large parts of their lives in groups. Within such groups, they need to find efficient ways of dividing available resources between them. This is often achieved by means of a dominance hierarchy, which in its most extreme linear form allocates a strict priority order to the individuals. Once a hierarchy is formed, it is often stable over long periods, but the formation of hierarchies among individuals with little or no knowledge of each other can involve aggressive contests. The outcome of such contests can have significant effects on later contests, with previous winners more likely to win (winner effects) and previous losers more likely to lose (loser effects). This scenario has been modelled by a number of authors, in particular by Dugatkin. In his model, individuals engage in aggressive contests if the assessment of their fighting ability relative to their opponent is above a threshold \(\theta \). Here we present a model where each individual can choose its own value \(\theta \). This enables us to address questions such as how aggressive should individuals be in order to take up one of the first places in the hierarchy? We find that a unique strategy evolves, as opposed to a mixture of strategies. Thus, in any scenario there exists a unique best level of aggression, and individuals should not switch between strategies. We find that for optimal strategy choice, the hierarchy forms quickly, after which there are no mutually aggressive contests.
Keywords
Resource holding potential Dominance hierarchy Expected payoff Stopping time Evolutionary game theory1 Introduction
Very often, animals that share the same territory engage in pairwise aggressive interactions leading to the formation of dominance hierarchies Hand (1986). Here we are interested in groups of animals that are meeting for the first time and have to engage in these aggressive interactions in order to divide their resources. In smaller groups of animals, the hierarchy tends to be linear where one individuals dominates all of the others, a second individual dominates all of the others in the group except the topranked individual and so on (see Addison and Simmel 1970; Barkan and Strahl 1986; Goessmann et al. 2000; Wilson 1971). In larger groups of animals, the hierarchy is more complex, where the position of especially lowerranked individuals may be unclear, e.g. in chimpanzees, baboons, hyenas (Kummer 1984; Möller et al. 2006, 2001; Widdig et al. 2001). Some animals are more aggressive than others, and the level of aggressiveness depends upon many factors such as experience, the value of winning the contest and resource holding potential (RHP) (see e.g. Blanchard et al. 1988; Blanchard and Blanchard 1977; Moss et al. 1994; Takahashi and Lore 1983; Taylor 1982).In our model, RHP is simply the ability of an individual to win an escalated contest (Parker 1974), abstracted away from any particular causal effect. In reality, there are a large number of elements that determine the RHP. Very broadly, these elements can be divided into physical attributes, such as size, age and physical strength (intrinsic factors), and psychological attributes, such as prior experience (extrinsic factors).
In more detail, there are a lot of results demonstrating a strong correlation between RHP and body size (Alexander 1961; Bridge et al. 2000; Lindström 1992). For example, it has been observed that larger animals are more aggressive towards smaller ones and that they have more chances of winning an encounter (Frey and Miller 1972; Knights 1987). However, other results show that such physical attributes are not the only important determinant of RHP. For example, Brown et al. (2006) showed that 37.5 % of the group in house crickets won aggressive interactions, even though they had smaller body size. In Hofmann and Schildberger (2001), bigger individuals lost 30 % of the aggressive interactions.
Prior experience as well can have an important effect on the RHP of an individual. For example, if an individual has won more fights than it has lost in the past, it may increase its potential to win in the future.
The aim of this paper is to explore the relationship between extrinsic factors, in particular prior experience, and hierarchy formation. Therefore, we assume in our model that all individuals have identical physical abilities, so that the outcome of an encounter is significantly determined by past experience (although our results depend upon only a mechanical updating of RHP after a contest, so it would allow for real physical as well as psychological changes, too). In particular, we consider socalled winner and loser effects. The winner effect occurs when winning a previous contest increases the chances that an animal wins a subsequent contest. The loser effect occurs when a previous loss similarly increases the chances of defeat in the next contest.
A number of authors have analysed the influence of winner and loser effects on dominance hierarchy formation (e.g. Bonabeau et al. 1999; Dugatkin 1997; Dugatkin and Dugatkin 2007; Hemelrijk 2000). The first models were developed by Landau (Landau 1951a, b). He demonstrated the importance of the winner and loser effects: only when extrinsic factors were considered in addition to intrinsic ones, did the resulting hierarchies resemble those found in nature. Landau considered populations where winner and loser effects were operating together, but there is evidence that some groups of animals experience either winner or loser effects only (Bakker et al. 1989; Bergman et al. 2003; Lindquist and Chase 2009; Schuett 1997). Dugatkin and Dugatkin (Dugatkin 1997; Dugatkin and Dugatkin 2007) developed a model where these effects were considered in isolation, or both to be present in a group of 4 individuals. In Dugatkin (1997), each individual could only knew its own RHP after a win or loss, but they did not have any information about their opponents strength, except at time \(t=1\). He predicted that when only the winner effect is at play, the emerging dominance hierarchies are linear and the strength of the winner effect is not important. Contrary, when only the loser effect is present, hierarchies where only the topranked individual is determined are found (the positions of the rest of the group stay unclear). When both winner and loser effects are present, nonlinear hierarchies emerged where only the first place, and sometimes the second place, was clear in the group. In Dugatkin and Dugatkin (2007), each individual was aware of their own RHP and they could make an imperfect estimate about their opponent’s RHP at each point in time. He concluded that overestimating or underestimating the opponent’s strength does not have any influence on linearity: in both cases, linear dominance hierarchies were established.
In Kura et al. (2015), we analysed the temporal dynamic and the average behaviour of dominance hierarchy formation for different combinations of winner and loser effects, using the model developed by Dugatkin (1997). We concluded that it is not necessary for a group of individuals to have perfect knowledge of each other’s RHP in order to establish a linear dominance hierarchy; only a little information about the current RHP estimation of an individual’s opponent is enough to establish a linear dominance hierarchy. We used different statistical measures such as the overlap between the distribution of the RHP of each individual over time to check for distinguishability between a pair of individuals. The index of linearity was used to measures how far from linearity each hierarchy is. Furthermore, we considered the question of how many fights are needed for a dominance hierarchy to be established, and we found that this number is relatively low.
In Dugatkin (1997) and Dugatkin and Dugatkin (2007) (as well as Kura et al. 2015), each individual had the same fixed level of aggression; they would retreat for the same excess of the number of wins over the number of losses. In this paper, we introduce gametheoretical elements in the form of aggressiveness level into this model. We assume that each individual can choose its own strategy, independent of their opponent’s strategy. We are particularly interested in determining the appropriate level of the aggression threshold and exploring whether a unique strategy, or mixture of strategies, emerges in the population considered. Our model setup allows us to answer questions such as under what circumstances should an individual fight more in order to establish a higher rank in the hierarchy and when should it retreat? We use a framework similar to the Hawk–Dove model Maynard Smith (1982), where an individual can choose to either fight or concede, with each individual making its choice simultaneously. When two individuals choose to fight, they engage in an aggressive interaction; the winner will increase its RHP by a factor \(1+V_{1}\), and the loser will reduce its RHP by a factor \(1C_{1}\). When one individual fights and the other concedes, the individual that chooses to fight increases its RHP by a factor \(1+V_{2}\) and the retreating individual has its RHP reduced by a factor \(1C_{2}\). In the case when both individuals retreat, they have their RHP multiplied by \(1C_{2}\). Individuals choose their own strategies, meaning whether to fight and or to concede in an aggressive interaction given their history of fights won and lost, from a range of possible strategies. For each of these possible strategies, we will determine the resulting expected payoff and conclude whether the chosen strategy is beneficial to the individual or not. We will analyse two cases: when each individuals choose a strategy that enables them to fight in all interactions, and when they choose strategies that enable them to fight until a certain point in time (based upon how many contests they have won or lost) and retreat afterwards. We will determine the evolutionarily stable strategies (ESSs) for this fighting game, where an ESS is a strategy that when played by almost all members of the population cannot be invaded by any other strategy. We will also calculate the possible stopping times of the game for different strategies and analyse the relationship between the stopping time and the difference of the number of wins and losses for an individual.
As explained above, individuals fight for more access to resources and we will investigate the effects of different payoff functions on the ESSs within our model. In particular, we compare payoffs which depend upon the level of resource an individual receives to those which depend upon the proportion of the overall resource that it receives. The latter payoff function is particularly appropriate when resources are scarce. Once the dominance hierarchy is established, it is easier for the group to divide resources between them: the higher the position in the hierarchy, the higher the payoff. The division of resources has been analysed by different authors (see e.g. Broom and Ruxton 2001; Keller and Reeve 1994). We will use the concept of reproductive skew (Broom et al. 2009; Keller and Reeve 1994; Reeve and Keller 2001; Shen and Reeve 2010; Vehrencamp 1983), which refers to the distribution of reproductive rights in a group of animals. We will use the term more generally to refer to how limited resources, and hence, payoffs (which are generally proportional to reproductive levels in evolutionary games) are divided among our group. When the reproductive skew is high, the division of resources is uneven with the highranking individuals obtaining more resources than the lowerranking ones (for example, see Drews 1993; Monnin and Ratnieks 1999; Rood 1980). In contrast, if the reproductive skew is low, the division of resources is even and all ranks of individuals have similar resource levels (see Brown 2014; Mangold et al. 2015). Further, we will explore the interplay between all three gametheoretical elements, \(V_{i}\), \(C_{i}\) and strategies \(\theta _{x}\), and analyse whether there is a general pattern for the ESS when the \(V_{i}\) and \(C_{i}\) are increased (or decreased). Additionally, we develop a simulation framework to investigate the effect of the group size on the level of aggression. We note that Andersen et al. (2004) developed an alternative optimisationbased model to analyse the effect of group size on aggression level and showed that the theoretical results obtained are supported by experimental data observed in domesticated pigs; we discuss this in Sect. 6. Lastly, we compare our theoretical results with experimental evidence which is rather different for different groups of animals such as birds, farmed animals or fish (see e.g. Andersen et al. 2004; Bilčık and Keeling 2000; Estévez et al. 1997; Estevez et al. 2007; Kotrschal et al. 1993; Nicol et al. 1999; Syarifuddin and Kramer 1996; Turner et al. 2001).
2 The Model
We assume a large population of social individuals living together in groups. At the beginning of the consideration, groups of size N are randomly formed, so that all individuals are members of a group and we analyse a specific group of N individuals. Each individual has an RHP value, which, as mentioned in the Introduction, is a measure of its ability to win an aggressive interaction (cf. Dugatkin 1997; Dugatkin and Dugatkin 2007) and which is altered by the outcome of each interaction. At the beginning, all individuals are assigned the same initial RHP, denoted by \({\hbox {RHP}}_{\hbox {initial}}\). We assume that all individuals know their own RHP and that of any opponent. In each round t (\(t=1,..., T\)), two individuals are randomly chosen to engage in an aggressive interaction, while the rest of individuals do not engage in any aggressive interactions. Through time, an individual’s RHP changes due to winning or losing (in reality, it will be mainly the extrinsic factors than change, but our model could cope with other eventualities equally well), while a win increases the RHP, a loss decreases it and each individual keeps track of the changes in their own RHP and that of its opponents. More precisely, suppose that at time t the two individuals pitted against each other are x and y. We denote by \({\hbox {RHP}}_{x,t}\) individual x’s RHP at time t. Individual x can decide to be aggressive or retreat once it has been chosen and this decision is based on the strategy \(\theta _{x} \ge 0\) which is its aggression threshold.
 1.Both individuals x and y decide to engage in an aggressive interaction and the probability that x wins is given byand consequently, individual y wins with a probability \(P_{y,x}(t)=1P_{x,y}(t)\).$$\begin{aligned} P_{x,y}(t)=\frac{{\hbox {RHP}}_{x,t}}{{\hbox {RHP}}_{x,t}+{\hbox {RHP}}_{y,t}}, \end{aligned}$$(2)
 2.
One individual engages in the aggressive interaction and the other retreats.
 3.
Both individuals decide not to fight (which is known as a double kowtow).
The aim of each member of the population is to maximise its payoff at time T. In the following, we assume that the payoff function is defined as the natural logarithm of the RHP (which corresponds to the situation of unlimited resources) but consider in Sect. 3.5 the effects of an alternative payoff function (which corresponds to the situation of limited resources). Now there are two main reasons for considering the natural logarithm of the RHP. Firstly, while we want to keep to Dugatkin’s terminology as much as possible, the multiplicative nature of how the RHP increases means that RHP values can become large very quickly. If we would assume the expected RHP as the payoff, then even a minuscule chance of winning enough contests to be the top individual would be worth almost any risk. Considering the logarithm means that winning (losing) any contest increases (decreases) the payoff by the same amount irrespective of the current RHP, which seems reasonable. Secondly, taking the natural logarithm of the RHP guarantees that the payoffs increase in precisely the same way as in evolutionary matrix games, and in particular the Hawk–Dove game, which we use as an analogy in this paper.
 1.
\(E[\theta _{x};\theta ^{N1}_{x}]>E[\theta _{y};\theta ^{N1}_{x}]\) or
 2.
\(E[\theta _{x};\theta ^{N1}_{x}]=E[\theta _{y};\theta ^{N1}_{x}]\) and \(E[\theta _{x},\theta ^{N2}_{x},\theta _{y}]>E[\theta _{y},\theta ^{N2}_{x},\theta _{y}]\),
 1.
\(E[\theta _{x},\theta _{x}]>E[\theta _{y},\theta _{x}]\) or
 2.
\(E[\theta _{x},\theta _{x}]=E[\theta _{y},\theta _{x}]\) and \(E[\theta _{x},\theta _{y}]>E[\theta _{y},\theta _{y}]\),
3 The TwoIndividual Model
For simplicity, in this section we consider groups of two individuals only. This will allow us to find some analytical results which will give us general insights into the dynamic of our model. We will then generalise to larger groups in Sect. 4.
3.1 Expected Payoffs When Players Always Fight (\(\theta _{x}=\theta _{y}=0\))
3.2 Individuals with General Strategies \(\theta _{x}\) and \(\theta _{y}\)
3.3 Stopping Time \(T_{s}\)
The expected payoff \(E[\ln ({\hbox {RHP}}_{x,T})]\) given by equation (21) depends on the stopping time \(T_s\). In this section, we explore the properties of \(T_s\) as defined by equation (20), in particular its distribution.
Note that it is possible for our model to generate one experience, a winner effect or a loser effect, without the other. For example, for \(V > 0\) and \(C=0\) we have a case when only the winner effect is in place. Tables 5 and 6 show the expected payoffs for different strategic values when \(V=0.1\) and \(C=0\). On the other hand, when \(C > 0\) and \(V=0\), illustrated by Tables 7 and 8, we have a case when only the loser effect is operating.
In the next section, we derive the distribution of \(T_s\) for the parameter constellation \(V=C=0.1\) (both winner and loser effect are influencing RHP).
3.4 Example: \(V=C=0.1\)
To illustrate the findings of the last sections, we consider an example by assuming the parameters \(V=0.1\), \(C=0.1\) and \(T=20\). In particular, we calculate the expected payoffs \(E[\ln ({\hbox {RHP}}_{x,20})]\) for different combinations of strategies \(\theta _{x}\) and \(\theta _{y}\), determine the unique ESS and derive the distribution of the stopping time \(T_s\).In this section and throughout the paper, we will assume that \({\hbox {RHP}}_{{\hbox {initial}}}=10\).
Firstly, we determine the representative strategies to \(k_{x}=1,2,3,4,5,6,7,8\) by using equation (24). Note that there is a range of strategies \(\theta _{x}\) that correspond to the same value of \(k_{x}\) and we take the middle one as described in Sect. 3.2. We obtain the following mappings (the same values apply for individual y as well).
\(k_{x}=1\Rightarrow \theta _{x,{\hbox {rep}}}=0.91\), \(k_{x}=2\Rightarrow \theta _{x,{\hbox {rep}}}=0.74\), \(k_{x}=3\Rightarrow \theta _{x,{\hbox {rep}}}=0.61\), \(k_{x}=4\Rightarrow \theta _{x,{\hbox {rep}}}=0.50\),
\(k_{x}=5\Rightarrow \theta _{x,{\hbox {rep}}}=0.41\), \(k_{x}=6\Rightarrow \theta _{x,{\hbox {rep}}}=0.33\), \(k_{x}=7\Rightarrow \theta _{x,{\hbox {rep}}}=0.27\), \(k_{x}=8\Rightarrow \theta _{x,{\hbox {rep}}}=0.22\).
The matrix of payoffs where each entry represent the expected payoff \(E[\ln ({\hbox {RHP}}_{x,T})]\) at time \(T=20\) [calculated by equation (21)] for different strategies \(\theta _x\) and \(\theta _y\)
\(k_{y}=1\) (\(\theta _{y}=0.91\))  \(k_{y}=2\) (\(\theta _{y}=0.74\))  \(k_{y}=3\) (\(\theta _{y}=0.61\))  \(k_{y}=4\) (\(\theta _{y}=0.50\))  \(k_{y}=5\) (\(\theta _{y}=0.41\))  \(k_{y}=6\) (\(\theta _{y}=0.33\))  \(k_{y}=7\) (\(\theta _{y}=0.27\))  \(k_{y}=8\) (\(\theta _{y}=0.22\))  

\(k_{x}=1\) (\(\theta _{x}=0.91\))  3.2000  2.8700  2.7300  2.6500  2.6000  2.5800  2.5600  2.5400 
\(k_{x}=2\) (\(\theta _{x}=0.74\))  3.4400  3.0600  2.8700  2.7600  2.7000  2.6600  2.6200  2.6000 
\(k_{x}=3\) (\(\theta _{x}=0.61\))  3.5000  3.1000  2.8900  2.7700  2.6900  2.6500  2.6100  2.6000 
\(k_{x}=4\) (\(\theta _{x}=0.50\))  3.5100  3.0800  2.8700  2.7400  2.6700  2.6200  2.5600  2.5600 
\(k_{x}=5\) (\(\theta _{x}=0.41\))  3.5000  3.0500  2.8400  2.700  2.6200  2.5700  2.5400  2.5200 
\(k_{x}=6\) (\(\theta _{x}=0.33\))  3.4600  3.0200  2.7900  2.6600  2.6000  2.5300  2.5000  2.4700 
\(k_{x}=7\) (\(\theta _{x}=0.27\))  3.4300  2.9900  2.7700  2.6200  2.5600  2.5000  2.4800  2.4500 
\(k_{x}=8\) (\(\theta _{x}=0.22\))  3.4200  2.9500  2.7300  2.6000  2.5100  2.4800  2.4400  2.4100 
Figures 2 illustrates that a pair of individuals will fight longer for higher values of \(k_{x}\) and \(k_{y}\). The reason behind this is that larger values of k correspond to smaller strategy values \(\theta \), and hence, equation (1) implies that the individuals will fight longer. In this example, one of the individuals x and y has started retreating before time T, for most of the possible cases. This means that observing 20 interaction would allow us to distinguish between the two individuals almost with certainty. As we increase the values of \(k_{x}\) and \(k_{y}\), the probability of retreating before \(T=20\) is decreased.
3.5 An Alternative Payoff Function
Division of resources for different values of k, when \(V=C=0.1\)
\(k_{y}=1\) (\(\theta _{y}=\)1)  \(k_{y}=2\) (\(\theta _{y}=0.7\))  \(k_{y}=3\) (\(\theta _{y}=0.6\))  \(k_{y}=4\) (\(\theta _{y}=0.5\))  \(k_{y}=5\) (\(\theta _{y}=0.4\))  \(k_{y}=6\) (\(\theta _{y}=0.35\))  \(k_{y}=7\) (\(\theta _{y}=0.27\))  \(k_{y}=8\) (\(\theta _{y}=0.23\))  

\(k_{x}=1\) (\(\theta _{x}=1\))  0.5000  0.4585  0.4417  0.4353  0.4344  0.4354  0.4360  0.4385 
\(k_{x}=2\) (\(\theta _{x}=0.7\))  0.5415  0.5000  0.4825  0.4777  0.4765  0.4777  0.4806  0.4825 
\(k_{x}=3\) (\(\theta _{x}=0.6\))  0.5583  0.5175  0.5000  0.4941  0.4928  0.4949  0.4973  0.4999 
\(k_{x}=4\) (\(\theta _{x}=0.5\))  0.5647  0.5223  0.5059  0.5000  0.4992  0.5013  0.5034  0.5066 
\(k_{x}=5\) (\(\theta _{x}=0.4\))  0.5656  0.5235  0.5072  0.5008  0.5000  0.5014  0.5046  0.5077 
\(k_{x}=6\) (\(\theta _{x}=0.35\))  0.5646  0.5223  0.5051  0.4987  0.4986  0.5000  0.5026  0.5054 
\(k_{x}=7\) (\(\theta _{x}=0.27\))  0.5640  0.5194  0.5027  0.4966  0.4954  0.4974  0.5000  0.5033 
\(k_{x}=8\) (\(\theta _{x}=0.23\))  0.5615  0.5175  0.5001  0.4934  0.4923  0.4946  0.4967  0.5000 
From Table 2, we find that \(\theta =0.4\) (corresponding to \(k=5\)) is the ESS. Comparing this result with the result obtained from Table 1, we notice that they differ; when using this alternative payoff function, we obtain \(k=5\) as the ESS, while for the original payoff function used in Sect. 3.4, the ESS is \(k=3\). This differences are related to the amount of the available resources, in particular whether they are plentiful or limited. We assume that for plentiful resources, the absolute RHP is more important, but for scarce resources shared between group members, the relative RHP is the key element. If an individual needs to maximise the RHP, then it should fight less compared to the situation where it needs to maximise the division of limited resources. In this latter case, the individual needs to be more aggressive so that it can win a greater share than its opponent, since “hurting” its opponent leads directly to improving its proportion in equation (27).
3.6 How the Expected Payoffs and the Division of Resources Change When Varying V and C
For the case when \(V=0\) and \(C > 0\), we expect the ESS to be the strategy where an individual retreats immediately. This is true when \(\ln ({\hbox {RHP}})\) is considered as the payoff function. When the alternative payoff function is used, we obtain \(k=1 (\theta =1)\) as the ESS (for \(C=0.1\)). Thus, in this case it is best to fight initially to potentially reduce the RHP of the opponent, as this increases the individual’s payoff function. On the other hand for \(C=0\) and \(V >0\), we obtain \(k\rightarrow \infty \) as the ESS. This is the expected result as since there is no cost for losing, it is best to fight until the end of the competition. When \( \frac{V}{C} \le 4\), we obtain lower values of \(\theta \) as an ESS for the alternative payoff function than for the payoff function given by \(\ln (RHP)\). This means that when resources are scarce, individuals need to be more aggressive in order to get a high payoff. For sufficiently high \(\dfrac{V}{C}\) ratio, (e.g. for \(\frac{V}{C} >4\)), we obtain the same value of \(\theta \) as an ESS for both payoff functions. The corresponding tables showing the expected payoffs for different combinations of \(k_{x}\) and \(k_{y}\) when V and C vary are given in Appendix.
4 The NIndividual Model
 S1 Initially, the N individuals can choose their strategies from the rangewith probability \(p(\theta =\theta _k)=\frac{1}{10}\), \(k=1,\ldots ,10\).$$\begin{aligned} \Theta =[\theta _1,\theta _2,\ldots ,\theta _{10}]=[0.1,0.2,\ldots ,1] \end{aligned}$$
Set \(i=0\).

S2.0 Set \(H=[0,0,\ldots ,0]\) (H has dimension 10) and \(j=0\).

S2.1 Each of the N individuals chooses a strategy \(\theta _{x_i},\ i=1,\ldots ,N\) according to the probability function \(p(\theta =\theta _k)\).

S2.2 Repeat the following for \(T_{{\hbox {max}}}=200\) times steps.
Randomly choose two individuals with their strategies \(\theta _{x_i}\) and \(\theta _{x_j}\) out of the N individuals and update their RHP according to Equations (3)–(6).
 S2.3 Update the vector H as followsSet \(j=j+1\). If \(j < \)10,000 go to S2.0 otherwise to S3.$$\begin{aligned} H(10\theta _{x_i})=H(10\theta _{x_i})+\ln ({\hbox {RHP}}_{x_i,200}),\quad i=1,\ldots ,N. \end{aligned}$$
 S3 Update probability function \(p(\theta =\theta _k)\) as followsSet \(i=i+1\). If \(i < \)10,000 go to S2.0 otherwise the simulation is finished.$$\begin{aligned} p(\theta =\theta _k)=\frac{H(10\theta _k)}{\sum \limits _{k=1}^{10}H(10\theta _k)}. \end{aligned}$$
4.1 Example: Population Size \(N=4\)
The ESS value of \(\theta \) for different combinations of V and C
\(C=0.025\)  \(C=0.05\)  \(C=0.075\)  \(C=0.1\)  \(C=0.125\)  \(C=0.15\)  

\(V=0.01\)  0.9400  0.9900  1.0000  1.0000  1.0000  1.0000 
\(V=0.02\)  0.8800  0.9000  0.9200  0.9500  0.9700  0.9900 
\(V=0.03\)  0.7000  0.8100  0.9000  0.9000  0.9200  0.9400 
\(V=0.04\)  0.6000  0.8000  0.8000  0.8700  0.9000  0.9100 
\(V=0.05\)  0.4900  0.7000  0.7900  0.8000  0.8000  0.9000 
\(V=0.06\)  0.4000  0.6200  0.7000  0.7700  0.8000  0.8000 
\(V=0.07\)  0.3700  0.6000  0.6900  0.7000  0.7600  0.8000 
\(V=0.08\)  0.3000  0.5000  0.6000  0.6900  0.7000  0.7000 
\(V=0.09\)  0.2900  0.5000  0.6000  0.6000  0.6500  0.6500 
\(V=0.1\)  0.2600  0.4400  0.5100  0.6000  0.6000  0.6900 
\(V=0.11\)  0.2100  0.4000  0.5000  0.5300  0.6000  0.6100 
\(V=0.12\)  0.2000  0.4000  0.5000  0.5100  0.6000  0.6000 
\(V=0.15\)  0.2000  0.3000  0.4000  0.5000  0.5000  0.5100 
\(V=0.18\)  0.1400  0.2900  0.3000  0.4000  0.4300  0.4400 
The ESS values for different combinations of V and C for \(N=2\) and \(N=4\)
\(N=2\)  \(N=4\)  

\(V=0.1, C=0.2\)  [0.53, 0.73]  0.9 
\(V=0,1, C=0.1\)  [0.55, 0.67]  0.6 
\(V=0.2, C=0.1\)  [0.32, 0.42]  0.5 
\(V=0.3, C=0.1\)  [0.23, 0.33]  0.35 
5 Comparison of Strategies
6 Discussion
In this paper, we have introduced gametheoretical elements to the winner–loser model developed in Dugatkin (Dugatkin 1997; Dugatkin and Dugatkin 2007). We considered a group of individuals that are characterised by their fighting ability score (their RHP) and a strategy \(\theta \) that indicates whether an individual would engage in an aggressive interaction or retreat. All individuals were assumed to possess the same RHP initially. We have developed a model that determines the expected payoff and ESS for different group sizes and payoffs, involving V and C, in such a population.
In the first part of this paper, we derived analytical results for a group of two individuals for the expected payoff and find the ESS, using \(\ln (RHP)\) as the payoff function, which correspond to situations with unlimited resources. In order to calculate the expected payoff for individual x with strategy \(\theta _{x}\), we first found the condition when this individual would retreat, represented by k. The variable k describes the critical difference between the number of wins and losses, below which individual x retreats. Given that a win increases the value of RHP, the value of k corresponds to the difference in RHP and thus only the individuals with a high RHP relative to its opponent risk engaging in an agonistic interaction to obtain more access to the available resources. We showed that there is a range of strategies \(\theta _{x}\) that correspond to the same value of k, meaning that they will give the same payoff. Furthermore different combinations of V and C yield different ranges of \(\theta _{x}\) for any given value of k.
We illustrated this analytical part with an example where we assumed \(V=C=0.1\). We found the expected payoff for different strategies \(\theta \ge 0\). In this case, we obtained a pure ESS which was achieved for \(k=3\), corresponding to the \(\theta \) range [0.55, 0.67]. Any strategy from this range gives the same payoff and is an ESS. We next varied V and C and saw the effect of this variations on the expected payoff and the ESS. As expected, if V is increased for a fixed C, the individuals will fight more, corresponding to lower values of \(\theta \). On the other hand, if C is increased for a fixed V, we get bigger values of \(\theta \) as an ESS. This means that individuals will fight less as C is increased.
We also used the idea of the reproductive skew (Broom et al. 2009; Keller and Reeve 1994; Reeve and Keller 2001; Shen and Reeve 2010; Vehrencamp 1983) to study how scarce resources are divided between a pair of individuals by using an alternative payoff function given in equation (27). When comparing the results with the ones obtained for the original payoff function, we observe smaller values of \(\theta \) as an ESS. This means that in this case individuals need to be more aggressive in order to obtain a larger share of the available resources.
While in our model, and in those of Dugatkin (1997) and Dugatkin and Dugatkin (2007), linear hierarchies are generally formed efficiently when (i) winner and loser effects are both present, (ii) only the winner effect or (iii) only the loser effect is present, the three models give clearly distinct predictions. With only the winner effect present, individuals in our model (for optimal strategy choice) and that of Dugatkin (1997) will continue fighting indefinitely, whereas in Dugatkin and Dugatkin (2007) individuals start fighting, but eventually contests cease. With only the loser effect present, individuals would give up immediately in our model (at least for the plentiful resources case defined by payoff function (11)), would give up after the first loss in the model of Dugatkin (1997), and would fight for some longer period in the model of Dugatkin and Dugatkin (2007). These differences in the results of the three models are rooted in the modelling assumptions. In Dugatkin (1997), there is no strategic choice and individuals do not know their opponent’s RHP; in Dugatkin and Dugatkin (2007), there is no strategic choice, but they do know their opponent’s RHP, and in our model, there is strategic choice and their opponent’s RHP is known. Thus, Dugatkin and Dugatkin (2007) can be thought of as an intermediate model between the other two. However, the predictions of our model are closer to that of Dugatkin (1997) than Dugatkin and Dugatkin (2007) and we would argue that these are more realistic.
Other authors have considered alternative gametheoretical models of dominance hierarchy formation. A good recent survey which raises some interesting questions and suggestions for further modelling is MestertonGibbons et al. (2016). We shall discuss two such models. Van Doorn and coworkers Doorn et al. (2003) analysed the evolution of dominance hierarchies by assuming that individuals are identical in ability throughout the time of their interaction, and so while their strategic choices depend upon past results, the actual probability of winning a contest depends upon the strategic choices of individuals, rather than their actual abilities. This is an example of what Maynard Smith Maynard Smith (1982) called an uncorrelated asymmetry (as opposed to a correlated asymmetry, as in our model). They found several evolutionary equilibria, one of them was the “dominance” equilibrium with the winner and loser effect where previous winners were more likely to take part in aggressive interactions and previous losers less likely to be aggressive. He also found a paradoxical equilibria where the higher position was occupied by the loser of an aggressive interaction than the winner. These results are very similar to the owner–intruder game Maynard Smith (1982) where paradoxical conventionbased outcomes can occur. They then extended this model to larger group sizes Doorn et al. (2003), where the individuals still had limited information about previous fights. Similar as in the twoplayer model, several evolutionary equilibria were found, one being with the winner and loser effect. The assumptions and outcomes are thus rather different to our model.
Fawcett and Johnstone (2010) developed a model to analyse the level of aggression where each individual differed in strength, but where they had no information about this difference. They predicted that the level of aggression is related to the amount of information that an individual has about prior contests. While the young individuals should be more aggressive as they are not sure about their fighting ability, the older one are not. They have knowledge of prior experience, and they retreat after a series of losses. Although the mechanisms differ, the actual way that the populations evolve is quite similar to ours. In their model, there are real differences between individuals, but the individuals start with no knowledge and learn over time; in our model, individuals have varying probabilities of being able to win a contest, which change (perhaps due to psychological factors) over time. In each case, after a time it is clear which individuals are the better ones, and the level of aggressive interactions declines, as more individuals play the more passive strategy. We note that in their model, the eventual division into mainly aggressive strong individuals and mainly passive weak individuals is dependent upon an intermediate number of strong/ weak individuals and that this divide would not happen for all population divisions.
In each of the strategic models discussed above (Fawcett and Johnstone 2010; Doorn et al. 2003 and Doorn et al. 2003, in addition to ours), individuals face a potentially long sequence of contests where they have two options at each step. Thus, in the same way as in games such as the classical iterated prisoner’s dilemma Axelrod (1984), there is a vast array of potential strategies. Each model reduces the dimensions of this strategy space in different ways . In the models of Doorn et al. (2003), Doorn et al. (2003), individuals were constrained to have a memory only of the latest interaction with an individual and so could base their play only on the results of this latest interaction (from the iterated prisoner’s dilemma “tit for tat” is such a strategy). Fawcett and Johnstone (2010) allow individuals to know their performance from all past contests, but allow them only to condition play on the total number of contests encountered, together with the number of wins in these contests. Our model behaves in a similar way to that of Fawcett and Johnstone (2010), basing strategy on the RHP, which in turn depends directly upon the number of won and lost contests of the participating individuals.
Similar results to those from our model concerning aggression levels have been found in experimental settings. Kotrschal et al. Kotrschal et al. (1993) performed a feeding experiment with greylag geese. Grained food was given in high, medium and low density. The geese were fed twice daily, and the level of aggression was recorded. They found a low number of agonistic interactions in the high food density setting and an increase in those aggressive interactions when the food density was decreased. Nie et al. Nie et al. (2013) conducted feeding experiments with varying levels of predation with root voles. They considered four treatments by combining different levels of predation and food supply (i.e. (no predation, food), (predation, food), (predation, no food), (no predation, no food)). They observed higher levels of aggressiveness in the groups treated with unfavourable conditions such as (predation, no food) compared to groups treated with (no predation, food). When the groups were treated with (predation, food) and (no predation, no food), the level of aggression observed was intermediate. These findings support our results that if resources are scarce, then an individual needs to be more aggressive.
An important concept related to the expected payoff is that of the stopping time. The stopping time is defined as the first time when one of the two individuals hits its stopping value of k. It gives a guideline for how many agonistic interactions we need to observe in a pair of individuals before one retreats. After hitting the stopping time, an individual would then always retreat afterwards. We showed in our example that twenty possible interactions is enough for an individual to retreat in almost all cases. Note that if \(T_{max}\) is relatively larger than the stopping time, the continued increase in the winner’s RHP after the stopping time is unrealistic. If, however, \(T_{max}\) is smaller than the stopping time, it is more difficult to distinguish between a pair of individuals in terms of their ranks in the hierarchy.
Analytical results can be derived for a group of two individuals, but for larger group sizes those derivations become effectively intractable. To explore the behaviour of larger group sizes, in particular to find the ESS, we developed in the second part of the paper a simulation approach. Analysing a group of four individuals, we found that the value of the ESS is increased when V is increased (for a fixed C), and by contrast, the value of the ESS is decreased when C is increased (for a fixed V). Comparing the values of ESS for a group of two individuals with the ones obtained for a group of four individuals leads to the conclusion that individuals should be less aggressive (i.e. fight less) in larger groups.
While this result is commonly observed in behavioural experiments, there are experimental settings leading to contradictory conclusions. For example, Nicol et al. Nicol et al. (1999) conducted a feeding experiment with Isa brown birds. They analysed the behaviour of the birds in groups of four different sizes (72, 168, 264 and 368). The birds were fed twice a day, and the number of aggressive pecking interactions were recorded. The results suggested a higher level of aggression in the smallest group (72) compared to the larger groups (168, 264, 368). Further, Anderson et al. Andersen et al. (2004) compared their model predictions (larger group sizes result in lower aggression levels) with results from an experiment with crossbred pigs. They considered three groups of 6, 12 and 24 pigs (which had not interacted with each other previously) which were put into pens and the space per individual was kept the same. There was one feeder per six pigs, and they were fed on ‘Format Start’ every morning. The aggressive interactions in each group were then recorded. It was observed that the level of aggression decreased with increasing group size. This result was also supported by further experiments Estevez et al. (2007), Estévez et al. (1997), Syarifuddin and Kramer (1996) Turner et al. (2001). However, Bilvci et al. Bilčık and Keeling (2000) observed the aggressive behaviour in a feeding experiment with groups of 15, 30, 60 and 120 Hisex white hens and noticed higher level of aggression in larger groups of birds than in the smaller ones.
Summarising, we presented a gametheoretical model which determines the evolutionarily stable aggression level in a populations of N individuals and different payoff functions, involving V and C, within a winner–loser framework. Within a group, we found that the population evolves to a unique aggression threshold, indicating that relative to their strength, all individuals adopt the same decision rule against whom to fight. Typically, the hierarchy is established quickly, with aggressive fights happening only in the early contests. Applied to realworld situations, this points to the crucial importance of the first few fights for hierarchy formation. Later fights only determine the position of lowerranked individuals. While higher values of C for losing an aggressive interaction (keeping the value of V constant) lead to lower aggression levels in the population, the reverse is true for increasing the value V for winning an aggressive interaction (keeping C constant): the higher the value of V, the higher is the aggression level in the population. Further, we predict lower aggression levels in larger populations. Our results are largely supported by experimental evidence so that we conclude that the introduction of gametheoretical elements to winner–loser models provides a further step towards a realistic description of aggressive interactions.
References
 Addison WE, Simmel EC (1970) The relationship between dominance and leadership in a flock of ewes. Bull Psychon Soc 15(5):303–305CrossRefGoogle Scholar
 Alexander RD (1961) Aggressiveness, territoriality, and sexual behavior in field crickets (Orthoptera: Gryllidae). Behaviour 130–223Google Scholar
 Andersen IL, Nævdal E, Bakken M, Bøe KE (2004) Aggression and group size in domesticated pigs, Sus scrofa: ’when the winner takes it all and the loser is standing small’. Anim Behav 68(4):965–975CrossRefGoogle Scholar
 Axelrod R (1984) The evolution of cooperation, vol 39. Basic Books, New YorkzbMATHGoogle Scholar
 Bakker TH, Bruijn E, Sevenster P (1989) Asymmetrical effects of prior winning and losing on dominance in sticklebacks (Gasterosteus aculeatus). Ethology 82(3):224–229CrossRefGoogle Scholar
 Barkan CI, Strahl AM (1986) Social dominance in communal Mexican jays Aphelocoma ultrumarina. Anim Behav 34(5):175–187CrossRefGoogle Scholar
 Bergman DA, Kozlowski CP, McIntyre JC, Huber R, Daws AG (2003) Temporal dynamics and communication of winnereffects in the crayfish, Orconectes rusticus. Behaviour 140:805–825CrossRefGoogle Scholar
 Bilčık B, Keeling LJ (2000) Relationship between feather pecking and ground pecking in laying hens and the effect of group size. Appl Anim Behav Sci 68(1):55–66CrossRefGoogle Scholar
 Blanchard D, Rodgers RJ, Hendrie CA, Hori K (1988) Taming of wild rats (Rattus rattus) by 5ht1a agonists buspirone and gepirone. Pharmacol Biochem Behav 31(2):269–278CrossRefGoogle Scholar
 Blanchard RJ, Blanchard DC (1977) Aggressive behavior in the rat. Behav Biol 21(2):197–224MathSciNetCrossRefGoogle Scholar
 Bonabeau E, Theraulaz G, Deneubourg JL (1999) Dominance orders in animal societies: the selforganization hypothesis revisited. Bull Math Biol 61(4):727–757CrossRefzbMATHGoogle Scholar
 Bridge AP, Elwood RW, Dick JTA (2000) Imperfect assessment and limited information preclude optimal strategies in malemale fights in the orbweaving spider Metellina mengei. Proc R Soc Lond B: Biol Sci 267(1440):273–279CrossRefGoogle Scholar
 Broom M, Cannings C, Vickers GT (1997) Multiplayer matrix games. Bull Math Biol 59(5):931–952CrossRefzbMATHGoogle Scholar
 Broom M, Koenig A, Borries C (2009) Variation in dominance hierarchies among groupliving animals: modeling stability and the likelihood of coalitions. Behav Ecol 20(4):844–855CrossRefGoogle Scholar
 Broom M, Ruxton GD (2001) A model of dominance and resource division among a group of animals of differing quality. Popul Ecol 43(3):213–220CrossRefGoogle Scholar
 Brown JL (2014) Helping Communal Breeding in Birds: Ecology and Evolution. Princeton University Press, PrincetonGoogle Scholar
 Brown WD, Smith AT, Moskalik B, Gabriel J (2006) Aggressive contests in house crickets: size, motivation and the information content of aggressive songs. Anim Behav 72(1):225–233CrossRefGoogle Scholar
 Drews C (1993) The concept and definition of dominance in animal behaviour. Behaviour 125(3):283–313CrossRefGoogle Scholar
 Dugatkin LA (1997) Winner and loser effects and the structure of dominance hierarchies. Behav Ecol 8(6):583–587CrossRefGoogle Scholar
 Dugatkin LA, Dugatkin AD (2007) Extrinsic effects, estimating opponents’ rhp, and the structure of dominance hierarchies. Biol Lett 3(6):614–616CrossRefGoogle Scholar
 Estevez I, Andersen I, Nævdal E (2007) Group size, density and social dynamics in farm animals. Appl Anim Behav Sci 103(3):185–204CrossRefGoogle Scholar
 Estévez I, Newberry RC, De Reyna LA (1997) Broiler chickens: a tolerant social system. Etologia 5:19–29Google Scholar
 Fawcett TW, Johnstone RA (2010) Learning your own strength: winner and loser effects should change with age and experience. In: Proceedings of the Royal Society of London B: Biological Sciences, page rspb20092088Google Scholar
 Frey DF, Miller RJ (1972) The establishment of dominance relationships in the blue gourami, trichogaster trichopter us (pallas). Behaviour 42(1):8–60CrossRefGoogle Scholar
 Goessmann C, Hemelrijk C, Huber R (2000) The formation and maintenance of crayfish hierarchies: behavioral and selfstructuring properties. Behav Ecol Sociobiol 48(6):418–428CrossRefGoogle Scholar
 Hand JL (1986) Resolution of social conflicts: dominance, egalitarianism, spheres of dominance, and game theory. Q Rev Biol 61(2):201–220CrossRefGoogle Scholar
 Hemelrijk CK (2000) Towards the integration of social dominance and spatial structure. Anim Behav 59(5):1035–1048CrossRefGoogle Scholar
 Hofmann HA, Schildberger K (2001) Assessment of strength and willingness to fight during aggressive encounters in crickets. Anim Behav 62(2):337–348CrossRefGoogle Scholar
 Keller L, Reeve HK (1994) Partitioning of reproduction in animal societies. Trends Ecol Evolu 9(3):98–102CrossRefGoogle Scholar
 Knights B (1987) Agonistic behaviour and growth in the European eel, Anguilla anguilla l., in relation to warmwater aquaculture. J Fish Biol 31(2):265–276CrossRefGoogle Scholar
 Kotrschal K, Hemetsberger J, Dittami J (1993) Food exploitation by a winter flock of greylag geese: behavioral dynamics, competition and social status. Behav Ecol Sociobiol 33(5):289–295CrossRefGoogle Scholar
 Kummer H (1984) From laboratory to desert and back: a social system of hamadryas baboons. Anim Behav 32(4):965–971MathSciNetCrossRefGoogle Scholar
 Kura K, Broom M, Kandler A (2015) Modelling dominance hierarchies under winner and loser effects. Bull Math Biol 77(6):927–952MathSciNetCrossRefzbMATHGoogle Scholar
 Landau HG (1951a) On dominance relations and the structure of animal societies: I. Effect of inherent characteristics. Bull Math Biophys 13(1):1–19MathSciNetCrossRefGoogle Scholar
 Landau HG (1951b) On dominance relations and the structure of animal societies: II. Some effects of possible social factors. Bull Math Biophys 13(4):245–262MathSciNetCrossRefzbMATHGoogle Scholar
 Lindquist WB, Chase ID (2009) Databased analysis of winnerloser models of hierarchy formation in animals. Bull Math Biol 71(3):556–584MathSciNetCrossRefzbMATHGoogle Scholar
 Lindström K (1992) The effect of resource holding potential, nest size and information about resource quality on the outcome of intruderowner conflicts in the sand goby. Behav Ecol Sociobiol 30(1):53–58CrossRefGoogle Scholar
 Mangold A, Trenkwalder K, Ringler M, Hödl W, Ringler E (2015) Low reproductive skew despite high malebiased operational sex ratio in a glass frog with paternal care. BMC Evol Biol 15(1):181CrossRefGoogle Scholar
 Maynard Smith J (1974) The theory of games and the evolution of animal conflicts. J Theor Biol 47(1):209–221MathSciNetCrossRefGoogle Scholar
 Maynard Smith J (1982) Evolution and the theory of games. cambridge University Press, CambridgezbMATHGoogle Scholar
 MestertonGibbons M, Dai Y, Goubault M (2016) Modeling the evolution of winner and loser effects: a survey and prospectus. Math Biosci 274:33–44MathSciNetCrossRefzbMATHGoogle Scholar
 Möller LM, Beheregaray LB, Allen SJ, Harcourt RG (2006) Association patterns and kinship in female IndoPacific bottlenose dolphins (Tursiops aduncus) of Southeastern Australia. Behav Ecol Sociobiol 61(1):109–117CrossRefGoogle Scholar
 Möller LM, Beheregaray LB, Harcourt RG, Krützen M (2001) Alliance membership and kinship in wild male bottlenose dolphins (Tursiops aduncus) of Southeastern Australia. Proc R Soc Lond B: Biol Sci 268(1479):1941–1947CrossRefGoogle Scholar
 Monnin T, Ratnieks FLW (1999) Reproduction versus work in queenless ants: When to join a hierarchy of hopeful reproductives? Behav Ecol Sociobiol 46(6):413–422CrossRefGoogle Scholar
 Moss R, Parr R, Lambin X (1994) Effects of testosterone on breeding density, breeding success and survival of red grouse. Proc R Soc Lond B: Biol Sci 258(1352):175–180CrossRefGoogle Scholar
 Nicol CJ, Gregory NG, Knowles TG, Parkman ID, Wilkins LJ (1999) Differential effects of increased stocking density, mediated by increased flock size, on feather pecking and aggression in laying hens. Appl Anim Behav Sci 65(2):137–152CrossRefGoogle Scholar
 Nie H, Yao M, Liu J (2013) Factors influencing aggression levels in root vole populations under the effect of food supply and predation. ISRN EcolGoogle Scholar
 Parker GA (1974) Assessment strategy and the evolution of fighting behaviour. J Theor Biol 47(1):223–243CrossRefGoogle Scholar
 Reeve HK, Keller L (2001) Tests of reproductiveskew models in social insects. Annu Rev Entomol 46(1):347–385CrossRefGoogle Scholar
 Rood JP (1980) Mating relationships and breeding suppression in the dwarf mongoose. Anim Behav 28(1):143–150CrossRefGoogle Scholar
 Schuett GW (1997) Body size and agonistic experience affect dominance and mating success in male copperheads. Anim Behav 54(1):213–224CrossRefGoogle Scholar
 Shen Sh, Reeve HK (2010) Reproductive skew theory unified: the general bordered tugofwar model. J Theor Biol 263(1):1–12MathSciNetCrossRefGoogle Scholar
 Syarifuddin S, Kramer DL (1996) The effect of group size on space use and aggression at a concentrated food source in blue gouramis, trichogaster trichopterus (Pisces: Belontiidae). Environ Biol Fishes 46(3):289–296CrossRefGoogle Scholar
 Takahashi LK, Lore RK (1983) Play fighting and the development of agonistic behavior in male and female rats. Aggress Behav 9(3):217–227CrossRefGoogle Scholar
 Taylor GT (1982) Urinary odors and size protect juvenile laboratory mice from adult male attack. Dev Psychobiol 15(2):171–186CrossRefGoogle Scholar
 Turner SP, Horgan GW, Edwards SA (2001) Effect of social group size on aggressive behaviour between unacquainted domestic pigs. Appl Anim Behav Sci 74(3):203–215CrossRefGoogle Scholar
 Van Doorn GS, Hengeveld GM, Weissing FJ (2003) The evolution of social dominance. I: twoplayer models. Behaviour 140(10):1305–1332CrossRefGoogle Scholar
 Van Doorn GS, Hengeveld GM, Weissing FJ (2003) The evolution of social dominance. II: multiplayer models. Behaviour 140(10):1333–1358CrossRefGoogle Scholar
 Vehrencamp SL (1983) A model for the evolution of despotic versus egalitarian societies. Anim Behav 31(3):667–682CrossRefGoogle Scholar
 Widdig A, Nürnberg P, Krawczak M, Jü Streich W, Bercovitch FB (2001) Paternal relatedness and age proximity regulate social relationships among adult female rhesus macaques. Proc Nat Acad Sci 98(24):13769–13773CrossRefGoogle Scholar
 Wilson EO (1971) The insect societies. Harvard University Press, Distributed by Oxford University Press, CambridgeGoogle Scholar
Copyright information
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.