Introduction

The size advantage model for sex changers is one of the most successful examples in sex allocation theory (Warner et al. 1975; Charnov 1982). The field observations, experimental manipulations, and theoretical modeling seem to be consistent with each other (Warner 1988; Kuwamura and Nakashima 1998). In typical cases, a single large individual and multiple subordinate individuals form a mating group in a protogynous species. The dominant individual is male, and the subordinate individuals are female. The dominant male engages in patrolling the group and drives out intruding competitors. The group is known as his harem. If the dominant male either dies or goes away, then the female with the highest rank in the female hierarchy immediately starts engaging in patrolling behavior (i.e., behavioral sex change), and after a relatively short period, it stops producing eggs and starts producing sperm, i.e., it becomes the new dominant male in the mating group.

Many aspects of sex change in coral fishes can be understood as traits that are formed at the evolutionary equilibrium of the game in which each individual is a player that maximizes its own fitness (or lifetime reproductive success). However, the standard theory of sex change does not discuss the time required for physiological modifications of the body, including the development and maturation of the gonad of the opposite sex (i.e., functional sex change). Functional sex change may require more than several weeks and sometimes several months to take place. Yamaguchi (2016) discussed the cost of reducing the time required for physiological changes considering sex hormonal dynamics (see also Yamaguchi and Iwasa 2018). In addition, there are species that keep the gonads of both sexes; however, only the gonad for the currently expressed sex is active, while the gonad for the opposite sex is inactive (bisexual gonad) (Sunobe and Nakazono 1993). Upon changing their social situation, individuals with bisexual gonads can change sex much more quickly than can species in which the gonad of the sex must be produced from scratch. In a previous paper, we studied the dynamic optimization model focusing on gonadal reconstruction for sex change (Yamaguchi and Iwasa 2017).

In most species of sex-changing fishes, the subordinates are female. However, there exist a few species in which the subordinate individuals are simultaneous hermaphrodites, which means that they have active gonads of both sexes and produce both eggs and sperm (Petersen and Fischer 1986, 1996; Petersen 1987, 1990; Leonard 1993). For example, Petersen and Fischer (1986) reported that certain coral reef fish, namely, Serranus baldwini, function as simultaneous hermaphrodites, with a male forming his harem in which multiple subordinate individuals stay and reproduce. When the harem male is removed, one of the hermaphroditic subordinates changes their sex to male. Another species of the same genus, Serranus fasciatus, is also a sex changer with hermaphroditic subordinates (Petersen 1987, 1990).

The presence of these species requires us to answer the following question: why do most species of sex-changing fishes have subordinates that are females? We hypothesize that the harem male may evict hermaphrodites out of his own territory to avoid possible sperm competition with those subordinates. Eviction is an example of reproductive suppression via aggression in group living animals, which has been studied extensively in many animal species (e.g., Ratnieks and Helanterä 2009; Sopinka et al. 2009; Cant et al. 2010; Cram et al. 2019). It is thus likely that eviction may also play a role in sex allocation decisions and group dynamics in social sex changers. However, because these hermaphrodites also lay eggs in the mating group, evicting all the hermaphroditic individuals would result in a reduction in the number of eggs laid; thus, the harem male must consider the costs and benefits of keeping/evicting these hermaphrodites.

The eviction behavior of the dominant male (or harem holder), in turn, should modify the fitness of subordinate individuals. By considering the risk of being evicted out of the group, subordinate individuals might avoid acting as competitors of the dominant male. This should result in subordinates that reduce their male investment and enhance their female investment. Hence, the sex allocation of subordinate individuals and the strategy of eviction by the harem holder affect each other in a closely related manner.

In this paper, we analyze this problem by constructing a simple game model. The players consist of a dominant male (harem holder) and subordinate individuals. The dominant male chooses whether to evict some subordinates to avoid sperm competition, while each subordinate individual chooses its own sex allocation considering the risk of being evicted by the dominant male. Mathematical analysis shows that in the evolutionarily stable state (ESS; Maynard Smith and Price 1973) or Nash noncooperative equilibrium (Nash 1950), when the summed male function of all the subordinates is smaller than that of the dominant male, then all the subordinates become females. Otherwise, all the subordinates become hermaphrodites. If this circumstance occurs, then large individuals invest an equal amount in male function, while small individuals invest in male function at a reduced level just to avoid the risk of eviction. This is consistent with the observation that hermaphrodites are more strongly female biased in sex allocation (Petersen 1987).

Model

We consider a mating group consisting of a single harem holder male and \(n\) subordinate individuals. Let \(i\) be an index of subordinates: \(i=\mathrm{1,2},3,...,n\). Let \({f}_{i}\) and \({m}_{i}\) be the female investment and male investment by subordinate \(i\), respectively. Thus, \({f}_{i}\) and \({m}_{i}\) can be regarded as the number of eggs and the amount of sperm produced, respectively.

Eviction of hermaphroditic subordinates by the dominant male

If subordinates are simultaneous hermaphrodites (\({m}_{i}>0\) and \({f}_{i}>0\)), then the harem holder may evict some hermaphrodite subordinates to avoid competition in regard to fertilizing eggs in the mating group. However, this would lead to a reduction in the number of eggs laid by the subordinates. The optimal strategy of eviction for the harem holder would be the one that achieves the maximum of his reproductive success:

$$\psi =\frac{\sum_{i\epsilon\Omega }{f}_{i}}{1+\alpha \sum_{i\epsilon\Omega }{m}_{i}}$$
(1)

where \(\Omega\) is a set of individuals that are allowed to stay in the group. Subordinate \(i\) is evicted if \(i\notin\Omega\), as \(\Omega\) represents the ones chosen to remain by the harem-holding male. Here, we assume that the relative success of the siring performance of subordinate \(i\) is proportional to the amount of male investment, with a proportionality coefficient, \(\alpha\). We may regard \(\alpha\) as the relative magnitude of the reproductive contribution by a subordinate individual compared to the dominant individual. Subordinate individuals may not be allowed to perform their mating behavior openly and can thus obtain their reproductive success by streaking, which is the act of joining the spawning pair midway through the rush and release sperm together with the gamete release of the pair (Marconato and Shapiro 1996); hence, the same amount of male investment by a subordinate might be less effective in fertilizing eggs than that by the dominant male. In such a case, the factor \(\alpha\) could be less than 1 (\(\alpha <1\)). However, the harem-holding male might have to spend a considerable amount of effort for the defense of his harem by patrolling and chasing out invading competitors; thus, the same amount of male investment by a subordinate individual might be more effective in fertilizing eggs than that by the harem holder (\(\alpha >1\)). Hence, the dominant individual and subordinates differ in two aspects: (1) The dominant individual is male and is able to evict others from the mating group if the action improves own fitness, but subordinates cannot perform such aggressive action. (2) The dominant male has the total reproductive resources different from each subordinate individual, with the relative magnitude of the latter being \(\alpha\). We also assume that the dominant individual performs eviction knowing the sex allocation of all subordinates (\({f}_{i}\) and \({m}_{i}\), for \(i=\mathrm{1,2},..,n\)), which would be plausible because they live in a close group.

According to the analysis shown in Online Resource 1 (Appendix A), we can derive the following result for the set of \(\Omega\) that is optimal for the harem holder.

If we renumber the subordinates according to the ratio \(\frac{{f}_{i}}{{m}_{i}}\) as follows:

$$\frac{{f}_{1}}{{m}_{1}}>\frac{{f}_{2}}{{m}_{2}}>...>\frac{{f}_{s}}{{m}_{s}}\ge \alpha \frac{\sum_{i\epsilon\Omega }{f}_{i}}{1+\alpha \sum_{i\epsilon\Omega }{m}_{i}}\ge \frac{{f}_{s+1}}{{m}_{s+1}}...>\frac{{f}_{n}}{{m}_{n}}$$
(2)

then the optimal set is \(\Omega =\left\{\mathrm{1,2},3,..,s\right\}\). This implies that the optimal \(\Omega\) consists of individuals with large \({f}_{i}/{m}_{i}\) only. The harem male should evict subordinates with small \({f}_{i}/{m}_{i}\).

Hence, the optimal solution for the harem holder is \(\Omega =\left\{\left.i\right|{f}_{i}>\theta {m}_{i}\right\}\) when the threshold value \(\theta\) is equal to \(\alpha \frac{\sum_{i\epsilon\Omega }{f}_{i}}{1+\alpha \sum_{i\epsilon\Omega }{m}_{i}}\). If \(\alpha\) is small, then the harem holder should accept subordinates that are hermaphrodite and thus possible competitors in regard to sperm competition.

The result of (2) is mathematically equivalent to the optimal diet theory (Pulliam 1974; Stephens and Krebs 1987).

This eviction behavior of the harem holder would force subordinate individuals to undergo sex allocation that satisfies \({f}_{i}\ge \theta {m}_{i}\). Let \({R}_{i}\) be the total reproductive resources of subordinate \(i\), which can be allocated to male function or female function. Because \({f}_{i}={R}_{i}-{m}_{i}\), this constraint is rewritten as \({m}_{i}\le {R}_{i}/\left(1+\theta \right)\).

Sex allocation of subordinates

Each subordinate chooses the allocation of resources to female and male functions to maximize their fitness. Here, they consider the risk of being evicted from the mating group by the harem male. Those individuals might differ greatly in the quantity of reproductive resources. The total reproductive resources of individual \(i\), \({R}_{i}\), depends critically on the body size, because a large individual has more resources that can be used for reproduction. In the paper, we call \({R}_{i}\) as the body size for short.

An evicted individual is assumed to lose the chance of engaging in breeding within the same season. If it were allowed by the harem holder of a new group, it might enjoy some reproductive successes. We here consider the situation in which the loss of fitness associated with eviction and moving between harems is large. For simplicity, the fitness enjoyed by the evicted individual in a new harem within the same breeding season is neglected.

The fitness of subordinate individual \(i\) is as follows:

$${\varphi }_{i}\left({m}_{i},{f}_{i}\right)={f}_{i}+\frac{\alpha {m}_{i}}{1+\alpha {m}_{i}+\alpha \sum_{j\in\Omega ,j\ne i}{m}_{j}}\left({f}_{i}+\sum \nolimits_{j\in\Omega ,j\ne i}{f}_{j}\right)$$
(3)

if the individual is not evicted by the harem holder. The first term indicates the investment in female function, i.e., the number of eggs produced by the individual. The second term indicates the number of eggs fertilized by the sperm of individual \(i\). The sum is calculated for all the individuals in the mating group \(\Omega\) except for the focal individual \(i\). We do not consider self-fertilization, because it does not take place in vertebrates except for mangrove killifishes, in which some offspring are produced from eggs fertilized by sperm, both produced by the same parent (Yamaguchi and Iwasa 2021).

Let \({K}_{i}=\sum_{j\in\Omega ,j\ne i}{f}_{j}\) and \({L}_{i}=\sum_{j\in\Omega ,j\ne i}{m}_{j}\), which are the values chosen by subordinates other than the focal individual. Hence, in choosing the optimal for subordinate \(i\), \({K}_{i}\) and \({L}_{i}\) must be handled as constants. These constants have suffix \(i\), which indicates the focal individual. In addition, we note that \({f}_{i}={R}_{i}-{m}_{i}\) (\(0\le {m}_{i}\le {R}_{i}\)); thus, Eq. (3) is rewritten as a function of \({m}_{i}:\)

$${\Phi }_{i}\left({m}_{i}\right)={R}_{i}-{m}_{i}+\frac{\alpha {m}_{i}\left({R}_{i}-{m}_{i}+{K}_{i}\right)}{1+\alpha {m}_{i}+\alpha {L}_{i}}$$
(4)

We search for the optimum of \({m}_{i}\) that maximizes \({\Phi }_{i}\left({m}_{i}\right)\) under the constraint \({0\le m}_{i}\le {R}_{i}/\left(1+\theta \right)\); the latter inequality originates from the avoidance of eviction by the harem holder (see Online Resource 1 (Appendix B) for explanations).

In Online Resource 1 (Appendix B), we calculate \(\frac{d{\Phi }_{i}}{d{m}_{i}}\left({m}_{i}\right)=0\), which is rewritten as follows:

$$1+\frac{\alpha {m}_{i}}{1+\alpha {m}_{i}+\alpha {L}_{i}}=\frac{\left(1+\alpha {L}_{i}\right)\alpha \left({R}_{i}-{m}_{i}+{K}_{i}\right)}{{\left(1+\alpha {m}_{i}+\alpha {L}_{i}\right)}^{2}}$$
(5)

In Online Resource 1 (Appendix B), we show that the second derivative of \({\Phi }_{i}\left({m}_{i}\right)\) is negative. Hence, if the solution \({m}_{i}\) of Eq. (5) is within \({0\le m}_{i}\le {R}_{i}/\left(1+\theta \right)\), it is the optimal for subordinate \(i\). If the solution of Eq. (5) is outside of the interval, then the optimal \({m}_{i}\) is a boundary value.

Evolutionarily stable state

By solving the combination of the optimal eviction strategy of a harem male and the optimal sex allocation by subordinates, we can calculate the evolutionarily stable strategy of this model.

When subordinates have equal amount of resources

When all subordinates have equal amounts of reproductive resources, then \({R}_{1}={R}_{2}={R}_{3}=...={R}_{n}=\overline{R }\). We expect all the subordinate individuals to take the same sex allocation, which is denoted by \(\left({m}^{*},{f}^{*}\right)\). The fitness of the harem holder is \(\psi =\frac{n{f}^{*}}{1+\alpha n{m}^{*}}\). In this case, the threshold value of eviction/noneviction is \(\alpha \frac{\sum_{i\epsilon\Omega }{f}_{i}}{1+\alpha \sum_{i\epsilon\Omega }{m}_{i}}=\alpha \frac{n{f}^{*}}{1+\alpha n{m}^{*}}<\frac{{f}^{*}}{{m}^{*}}\), which implies that no subordinate individuals should be evicted. Eviction can occur only when the variation in reproductive resources among subordinates is sufficiently large.

By setting \({m}_{i}={m}^{*}\), \({L}_{i}=\left(n-1\right){m}^{*}\), \({K}_{i}=\left(n-1\right)\left(\overline{R }-{m}^{*}\right)\), \({R}_{i}=\overline{R }\), Eq. (5), i.e., the condition for the sex allocation optimal for subordinates, leads to the following:

$$\left(1+\alpha \left(n+1\right){m}^{*}\right)\left(1+\alpha n{m}^{*}\right)=\left(1+\alpha {\left(n-1\right)m}^{*}\right)\alpha n\left(\overline{R }-{m}^{*}\right)$$
(6)

Both sides of Eq. (6) are quadratic functions of \({m}^{*}\). By examining the slope and curvature, we can derive that if \(1>\alpha n\overline{R }\), there is no positive solution to Eq. (6); however, if \(1<\alpha n\overline{R }\), there is a single positive solution \({m}^{*}\) that satisfies \(0<{m}^{*}<\overline{R }\).

Hence, we can conclude the following:

  1. 1.

    If \(\alpha n\overline{R }\le 1\), then all subordinates are females.

  2. 2.

    If \(\alpha n\overline{R }>1\), then all subordinates are hermaphrodites, and the sex allocation is determined by \({m}^{*}\) as the solution of Eq. (6) (\(0<{m}^{*}<\overline{R }\)). Because \(\overline{R }-{m}^{*}>0\), female investment and male investment are both positive.

Inequality \(\alpha n\overline{R }\le 1\) implies that the summed male function of all the subordinates, even if they all became pure males, is smaller than that of the harem holder. In such a condition, all the subordinates are females (not hermaphrodites) in the ESS. In contrast, if the subordinates have a summed potential male function that is greater than that of the harem male, then all the subordinates are hermaphrodites in the ESS, and the sex allocation rate is determined by Eq. (6).

When subordinates differ in reproductive resources

Now, we consider the case in which reproductive resources \({R}_{i}\) may differ between individuals. We introduce \(\widehat{K}={\sum }_{j\in\Omega }{f}_{j}={\sum }_{j\in\Omega }\left({R}_{j}-{m}_{j}\right)\) and \(\widehat{L}={\sum }_{j\in\Omega }{m}_{j}\), which are independent of suffix \(i\), thereby indicating that they are common to all the subordinates. Equation (5) is obtained from \(\frac{d{\Phi }_{i}}{d{m}_{i}}\left({m}_{i}\right)=0\). Using these new symbols, \({m}_{i}\) satisfying Eq. (5) is rewritten as \({m}_{i}=\overline{m }\), where:

$$\overline{m }=\frac{\left(1+\alpha \widehat{L}\right)\left(\alpha \widehat{K}-1-\alpha \widehat{L}\right)}{\alpha \left(1+\alpha \widehat{L}+\alpha \widehat{K}\right)}$$
(7)

This value is independent of \({R}_{i}\) and is common to all subordinates. According to Online Resource 1 (Appendix C), the fitness function \({\Phi }_{i}\left({m}_{i}\right)\) has a negative second derivative. Hence, if the solution of Eq. (7) satisfies \(0\le \overline{m}\le {R }_{i}\), the stationary solution \({m}_{i}=\overline{m }\) is the optimal value that achieves the maximum of \({\Phi }_{i}\left({m}_{i}\right)\). However, \(\overline{m }>{R}_{i}\) may hold for small individuals. For these individuals, the optimum is \({m}_{i}={R}_{i}\), which indicates that the individual should be pure male. Note that this calculation does not consider the risk of eviction by the harem male. According to the analysis in the last section, the dominant male evicts subordinate individuals with \({m}_{i}>{R}_{i}/\left(1+\theta \right)\). Small subordinate individuals must consider this risk when determining their optimal sex allocation. Hence, their optimal value is \({m}_{i}\approx {R}_{i}/\left(1+\theta \right)\) or slightly smaller than \({R}_{i}/\left(1+\theta \right)\). This argument is explained in Online Resource 1 (Appendix C) in more detail.

Considering the optimal sex allocation condition, we have two situations.

First, we examine the ESS condition for the solution in which all subordinates are females. In this solution, \(\widehat{K}=\sum_{j\in\Omega }{R}_{j}\) and \(\widehat{L}=0\) hold. Evolutionary stability requires \(\overline{m }\le 0\), which holds if \(\alpha \sum_{j\in\Omega }{R}_{j}-1\le 0\). In such a case, the fitness \({\Phi }_{i}\left({m}_{i}\right)\) is a decreasing function of \({m}_{i}\) for \({m}_{i}>0\); hence, the optimal is \({m}_{i}=0\). Because the condition \(\alpha \sum_{j\in\Omega }{R}_{j}-1\le 0\) is independent of suffix \(i\), this holds for all the individuals \(i\). Inequality \(\alpha \sum_{j\in\Omega }{R}_{j}\le 1\) indicates that the sum of the male mating ability of all the subordinates is smaller than that of the harem-holding male. Note that both \(\alpha\) and \({R}_{j}\) are defined as values relative to the harem male.

In contrast, if \(\alpha \sum_{j\in\Omega }{R}_{j}-1>0\) holds, then function \({\Phi }_{i}\left({m}_{i}\right)\) has a peak at a positive value. There is a constraint given by the risk of eviction by the harem holder. According to Eq. (2), the latter is given as \({f}_{i}/{m}_{i}=\left({R}_{i}-{m}_{i}\right)/{m}_{i}>\theta\), where \(\theta \approx \alpha {\sum }_{i\in\Omega }\left({R}_{i}-{m}_{i}\right)/\left(1+\alpha {\sum }_{i\in\Omega }{m}_{i}\right)\). This constraint is stricter than the other one, i.e., \(0\le {m}_{i}\le {R}_{i}\). After some calculations that are explained in Online Resource 1 (Appendix C), we can derive the optimal as follows:

  1. 1.

    If \({R}_{i}>\left(1+\theta \right)\overline{m }\), then the optimal is \({m}_{i}=\overline{m }\).

  2. 2.

    If \({R}_{i}<\left(1+\theta \right)\overline{m }\), then the optimal is \({m}_{i}=\frac{{R}_{i}}{1+\theta }\).

These equations can be summarized as \({m}_{i}=min\left[\overline{m },\frac{{R}_{i}}{1+\theta }\right]\). The value of the eviction threshold \(\theta\) and male allocation for large subordinate individuals \(\overline{m }\) are dependent on the male allocation of all subordinates \({\left\{{m}_{i}\right\}}_{i\in\Omega }\), which in turn depends on \(\theta\) and \(\overline{m }\). To know these values that satisfy all the necessary conditions, we adopt an iterative method, as explained in Online Resource 1 (Appendix C).

Figure 1 illustrates the ESS in a group of subordinates with different \({R}_{i}\). Horizontal and vertical axes represent male investment \({m}_{i}\) and female investment \({f}_{i}\), respectively. Circles indicate the sex allocation of subordinates. Figure 1a shows the case when \(\alpha \sum_{j\in\Omega }{R}_{j}\le 1\), i.e., the harem male is very effective in regard to male function. In the ESS, all the subordinates are pure female and thus produce no sperm (\({m}_{i}=0\)). Figure 1b shows the case when \(\alpha \sum_{j\in\Omega }{R}_{j}>1\), which implies that the summed mating ability of pure male subordinates would be larger than that of the harem male. In such a case, all the subordinates are hermaphrodites in the ESS (\({m}_{i}>0\) and \({f}_{i}>0\)). Large individuals (with large \({R}_{i}={m}_{i}+{y}_{i}\)) have equal male investment \({m}_{i}=\overline{m }\), indicated by circles all on a vertical line. Small individuals (with small \({R}_{i}\)) are constrained by the risk of eviction, and their sex allocation are on a line passing through the origin, \({f}_{i}=\theta {m}_{i}\).

Fig. 1
figure 1

The ESS sex allocation of subordinate individuals. Horizontal and vertical axes indicate male investment \({m}_{i}\) and female investment \({f}_{i}\), respectively. Circles are the sex allocation of subordinates with different total reproductive resources \({R}_{i}\). a When \(\alpha \sum_{j\in\Omega }{R}_{j}\le 1\) holds, all subordinates are female (\({m}_{i}=0\)), as indicated by the fact that all circles are on the \(y\)-axis. b When \(\alpha \sum_{j\in\Omega }{R}_{j}>1\) holds, all subordinates are hermaphrodites (\({m}_{i}>0, {f}_{i}>0\)). Large subordinate individuals have the same allocation to male function (\({m}_{i}=\overline{m }\)), as indicated that circles are on a vertical line where \(\overline{m }\) is given by Eq. (7). They differ in the female function that increases with body size \({R}_{i}\). They are not constrained by the eviction risk. In contrast, small subordinate individuals take the sex allocation just to avoid eviction risk. A broken line indicates the constraint \(f=\theta m\), where \(\theta\) is given by \(\theta =\alpha \sum_{j\in\Omega }\left({R}_{j}-{m}_{j}\right)/\left(1+\alpha \sum_{j\in\Omega }{m}_{j}\right)\). In the region below the broken line, the subordinate will be evicted (see the text)

Full size image

Figure 2 illustrates how the investment in male and female functions changes with the total amount of resources of different subordinate individuals. Open circles and closed circles indicate female investment and male investment of different individuals in the same mating group. Figure 2a illustrates the case in which all subordinates choose to be pure females. The situation corresponds to the one in Fig. 1a, which occurs if \(\alpha {\sum }_{i\in\Omega }{R}_{i}<1\). In contrast, Fig. 2b illustrates the case when all subordinate individuals are hermaphrodites. The situation corresponds to the one in Fig. 1b, which occurs if \(\alpha {\sum }_{i\in\Omega }{R}_{i}>1\).

Fig. 2
figure 2

Female allocation and male allocation for different values of reproductive resources. Horizontal axis is for \({R}_{i}\). Open and closed circles indicate female investment (\({f}_{i}\)) and male investment (\({m}_{i}\)), respectively. a When \(\alpha \sum_{j\in\Omega }{R}_{j}\le 1\) holds. All subordinates are females (\({m}_{i}=0, {f}_{i}={R}_{i}\)). b When \(\alpha \sum_{j\in\Omega }{R}_{j}>1\) holds, all subordinates are hermaphrodites (\({m}_{i}>0, {f}_{i}>0\)). For large subordinates (large \(R\)), male investment is the same but female investment increases with \({R}_{i}\). For small individuals (small \(R\)), both \({m}_{i}\) and \({f}_{i}\) increase with \({R}_{i}\). These two cases correspond to a and b in Fig. 1. See the explanation given in Fig. 1

Full size image

Figure 3 illustrates how the ESS sex allocation depends on the value of \(\alpha\), which is the relative siring ability of the subordinates. The horizontal axis represents \(\alpha\). When \(\alpha\) is small (i.e., \(\alpha <1/{\sum }_{i\in\Omega }{R}_{i}\)), all the subordinates are females. In contrast, when \(\alpha\) exceeds the threshold (\(\alpha >1/{\sum }_{i\in\Omega }{R}_{i}\)), all the individuals invest in some male function as well as in some female function (i.e., they are hermaphrodites). Different curves represent different individuals with different sizes of \({R}_{i}\). If \(\alpha\) exceeds the threshold by a small amount, then the male investments made by different individuals are the same (\({m}_{1}={m}_{2}={m}_{3}\)). As \(\alpha\) increases further, their level increases; however, small individuals come to invest less in male function compared to larger individuals.

Fig. 3
figure 3

Investment in male function (e.g., sperm production) by different subordinate individuals. The horizontal axis indicates the relative siring ability of subordinates, \(\alpha\). When \(\alpha {\sum }_{i\in\Omega }{R}_{i}<1\) holds, all the subordinates are females. In contrast, when \(\alpha {\sum }_{i\in\Omega }{R}_{i}>1\), all the individuals have some level of investment in male function. Different curves are for different individuals with varying sizes of \({R}_{i}\). There are three individuals differing in the total reproductive investment (\({R}_{1}>{R}_{2}>{R}_{3}\)). A triangle indicates a threshold value of \(\alpha\). When \(\alpha\) is smaller than the threshold, all individuals are pure female (\({f}_{i}={R}_{i}\), \({m}_{i}=0\)). When \(\alpha\) is larger than the threshold, all individuals are hermaphrodite (\({f}_{i}>0\), \({m}_{i}>0\))

Full size image

Discussion

In this paper, we asked whether subordinate individuals in sex-changing fishes are forced to be female. We examined the cases in which the dominant male can chase out hermaphroditic subordinate individuals that are strong competitors in regard to fertilizing eggs. On the other hand, those hermaphrodites also lay eggs themselves and contribute to the reproductive success of the harem male. To maximize his own fitness, the dominant male must carefully choose who to evict and who to allow to remain in the harem.

Our analysis based on game theory shows the following results. First, eviction behavior should never be beneficial to the harem male if subordinates have the same amount of reproductive resources. Second, even if the variation among subordinate individuals concerning reproductive resources is large, eviction should not be profitable to the harem male if the total sum of the siring ability of all the subordinates combined is smaller than that of the dominant male. This is because, in such a situation, all the subordinates choose to be female due to their own reproductive success in the absence of coercion. The dominant male finds it profitable to engage in eviction only if the variation in size among the subordinates is large and if the total sum of their siring potential exceeds that of the harem male. Even in that case, eviction should target only small subordinate individuals and not large individuals because large subordinates should choose to adopt female-biased sex allocation even in the absence of coercion.

Hence, we may conclude that the rarity of hermaphroditic subordinates in sex changers is not very likely to be explained by the risk of eviction by the dominant male. Rather, it should be the outcome of the free choice of sex allocation made by the subordinates. This supports the size-advantage model and the observation that fitness is maximized for each sex depending on the size and mating system (Benvenuto et al. 2017). The fact that in most sex-changing species all subordinates are female suggests that the summed siring ability of all the subordinate individuals in the mating group is likely to be smaller than that of dominant males. If this is true, then the subordinates will choose to be female in the absence of coercion by the dominant male.

Hermaphroditic sex changers have territories that are not separated clearly from neighboring territories; thus, subordinates in one territory can gain reproductive success in a nearby territory by engaging in streaking behavior (Fischer and Petersen 1987). Leonard (1993) pointed out that eviction behavior is shown by the harem-holding male toward individuals from outside of the same mating group, which enter to steal food within the territory. Most likely, we need to examine the effect of the spatial structure of the breeding population, especially the possibility of entering territories in neighboring areas.

Possibility of response to the phenotype of the opponent

In the game model discussed in the present paper, we assumed that the dominant male could observe the sex allocation of each subordinate individual and that the male performs eviction behavior based on this information. In contrast, while subordinate individuals adjust their sex allocation to the general level of eviction behavior of the dominant male, the subordinates cannot adjust the eviction level intended by the dominant male in the group. Namely, we assumed that the dominant male responds to the choice of subordinates (i.e., \({m}_{i}\) and \({f}_{i}\)), but subordinates cannot change their sex allocation in response to the choice of the dominant male (i.e., \(\theta\)). We adopted this assumption because once subordinates are evicted from the group, their sex allocation cannot be adjusted.

However, it might be possible that the dominant male could repeatedly harass individuals well before the eviction event and that a subordinate might thus adjust their sex allocation after being the victim of such harassment, thereby reducing their chance of eviction. If so, the subordinates may be able to respond to the possibility of eviction given by the dominant male. We did not adopt this assumption in our analysis because if this were the case, then all harem holders should harass subordinates to force them to be pure females, which would not be consistent with the fact that there are sex-changing species with hermaphroditic subordinates.

In some sex-changing fishes, when a male that is stronger than the harem holder joins the territory, the previous harem-holder male starts to adopt subordinate behaviors, and after several weeks, it goes back to being female (bidirectional sex change; Hobbs and Munday 2004; Kuwamura et al. 2011; Sawada et al. 2017). In the meantime, the dominant male, i.e., the new harem holder, will wait for this reverse sex change to complete. Hence, it is not surprising that hermaphroditic subordinates may also stay in the territory if they start to change their reproductive sex allocation. Yamaguchi and Iwasa (2015) showed that the game model with phenotype adjustment gives very different predictions from those of the game model without phenotypic adjustment. Hence, it is important to know if harassment by the dominant male could modify the sex allocation of subordinates in the field.

Predictions of the model

This simple model has predictions that should be tested by observations or experimental manipulation in the field. First, in most species of sex-changing fishes in coral reefs, the subordinates are all females (polygynous subordinates). The model predicts that inequality \(\alpha \sum_{j\in\Omega }{R}_{j}\le 1\) should be satisfied for these sex-changing fishes because if the opposite inequality held, then all subordinates would be hermaphrodites in the ESS. This clear prediction needs to be tested in the field.

Second, in some sex-changing species of coral reefs (Petersen and Fischer 1986, 1996; Petersen 1987, 1990), the subordinates are hermaphrodites. The model predicts that, for these species, the summed male function of all the subordinates should be greater than that of the dominant male (i.e., \(\alpha \sum_{j\in\Omega }{R}_{j}>1\) holds). This is certainly a testable prediction. In five of the six species in the genus Serranus, additional conspecifics have been observed to join the spawning pair. These individuals are presumed to release sperm and are called streakers, based on behavioral observations and analogous behavior in separate-sexed species (Fischer 1984; Petersen and Fischer 1986; Petersen 1987, 1991). This suggests that subordinate individuals might have more opportunity to succeed in participating in siring activity than other sex-changing fishes. Studying habitat structure, mating behavior, body size difference, and the number of subordinates per harem may be useful in clarifying this point.

Third, among those species with hermaphroditic subordinates, the theory predicts that individuals that are larger than the threshold size should make an equal investment in the male function and that their female investment should strongly depend on their size. In contrast, individuals smaller than the threshold should make lower male investment to avoid eviction risk. A field study reported that sexual investment is reported to be more strongly female biased among large subordinate individuals than small individuals (Petersen 1987), which is consistent with the prediction of the model. Quantitative measurements of sex allocation of these subordinate individuals would be desirable.

Fourth, eviction behavior and harassment behavior by the dominant male focus on small individuals that have male-biased sex allocation. Harassment behavior has been reported in some studies (Fischer and Petersen 1987); however, these studies did not clearly state the body size of the victims. In the face of the risk of being evicted, small subordinates may engage in sex allocation that is less than the value caused by the eviction behavior of the dominant male. As a result, eviction behavior may not appear very frequently, although the possibility of eviction behavior shapes the sex allocation of subordinates.

The situation studied in this paper is an example in which subordinates and dominants are in conflict over the sex status of subordinates. A question is whether the aggression by the dominant can exert control over the sex allocation decisions of subordinates. The outcome of the conflict of interest between individuals depends on many aspects of the game. This is consistent with the conclusion of theoretical studies on the co-evolution of social cohesion and eviction, including relatedness-dependent responses to aggression (Thompson et al. 2017), the role of ecological constraints and outside options for subordinates (Johnstone and Cant 1999; Cant and Johnstone 2009), and eviction as a tool to achieve optimal group sizes (Stephens et al. 2005). These demonstrate that many different conditions and factors of these models have been identified as important. To analyze the situation accurately, we need to specify the variables that can be controlled by each player, the order of decision-making of the players, and the information available at the time of their decision-making. The analysis of the multi-stage game model indicated that the effect was rather limited in the situation analyzed in this paper.

Considering the transfer of the evicted individual and the possibility of future reproductive performance in a new harem within the same breeding season would be an interesting theme of future theoretical study. The possibility of enjoying reproductive success in a different harem might affect the optimal behavior of subordinates in the original harem. To model such an effect, we need to consider many additional elements, including the body size distribution of individuals over the whole population and the cost associated with moving between harems. Probably, dynamics optimization formalism would be useful, as adopted in Sawada et al. (2017).

Some readers might wonder why we focus on the rare case of hermaphroditic sex-changing fishes, while many sex-changing fishes have subordinates that are predominantly purely female. We may learn a lesson from the history of evolutionary biology in the twentieth century. The game theoretic explanation for the 1:1 sex ratio, commonly called the Fisherian sex ratio (Fisher 1930), has been known among the evolutionary biology community since the nineteenth century (Düsing 1884; see Edwards 2000). However, many biologists did not realize its importance until William D. Hamilton showed that parasitic wasps have an extraordinary female-biased sex ratio, which, together with his explanation of local mate competition (Hamilton 1967), led to the Hamiltonian revolution in evolutionary biology. Asking questions about very minor examples, such as the biased sex ratio of parasitic wasps, may clarify the basic principle that governs most cases. Thus, we are very much concerned about the rare example of hermaphroditic sex changers. We believe that thinking about these rare species will reveal why subordinates are females in most sex changers.