Introduction

The extent to which individuals engage in inbreeding and outbreeding has a profound influence on evolution because it affects how genetic variation is structured and maintained (Charlesworth 1992, 2003). The evolutionary balance between the benefits of inbreeding and outbreeding to an individual has long been a focus of theory and empirical work in seed plants, and helps explain patterns of variation in plant reproductive traits, life history, and spatial genetic structure (Barrett et al. 1996). Yet, despite the often referenced overlap in morphology, clonal reproduction, and ecology of plants and sessile marine invertebrates (Jackson et al. 1985), the potential to apply lessons learned from the study of plant inbreeding and outbreeding has only rarely been explored for marine organisms (but see Grosberg 1987; Knowlton and Jackson 1993; Brazeau et al. 1998; Carlon 1999; Hoare and Hughes 2001; Sherman 2008; Hughes et al. 2009; Carlon and Lippe 2011). Where inbreeding and outbreeding have been a focus of interest in marine invertebrate research, it has primarily been in the context of the negative effects of inbreeding and its consequences for breeding programs and species conservation (Ibarra et al. 1995; Knowlton 2001; Baums 2008; Anderson and Hedgecock 2010), or the role of mating between individuals from different populations in the evolution of reproductive isolation (Burton 1986; Ellison and Burton 2008).

Three common perspectives have contributed to limiting the exploration of the causes and consequences of inbreeding in the study of marine invertebrates. First, the historical view that marine populations are large, open, and well mixed suggests that opportunities for inbreeding will be uncommon. Second, the misperception that plant mating system theory is only relevant to organisms that can self-fertilize suggests it will rarely be applicable to animals with separate sexes (but see Kokko and Ots 2006; Puurtinen 2011; Duthie and Reid 2016). Third, the view that inbreeding is generally deleterious focuses attention on the evolution of mechanisms to avoid it and diminishes consideration of the less well-recognized potential benefits of inbreeding. However, recent empirical findings and developments in theory have called these historical perceptions into question.

A recent survey of estimates of inbreeding (FIS) of sessile and sedentary marine invertebrates revealed a distribution quite similar to that for seed plants (Olsen et al. 2020). Although FIS is typically greater for hermaphroditic animals, estimates for taxa with separate sexes span nearly the full range from − 0.05 to 0.95 with a mode at 0.05, indicating that appreciable inbreeding in marine invertebrates occurs via mating between close relatives (biparental inbreeding). Associations reported between estimated FIS and traits of marine invertebrates (Olsen et al. 2020) support a role for inbreeding in driving evolutionary transitions between hermaphroditism and separate sexes, direct development and multiphasic life cycles, and external and internal fertilization. Collectively, these results suggest that inbreeding could influence the evolution of many traits of marine invertebrates, just as has been demonstrated for plants.

Other empirical findings are also consistent with the occurrence of inbreeding in sessile and sedentary marine invertebrates. Data collected over the past few decades have revealed surprisingly strong evidence for genetic structure in populations of some benthic marine invertebrates (reviewed by Hellberg 2009; Weersing and Toonen 2009), including recent findings that kin may settle in close proximity even after substantial larval dispersal (Veliz et al. 2006; Iacchei et al. 2013; Aglieri et al. 2014; Burgess et al. 2016). Given fine-scale spatial genetic structure, biparental inbreeding is potentially common for both hermaphrodites and taxa with separate sexes, and is expected to have many of the same costs and benefits as selfing (Lehtonen and Kokko 2015; Porcher and Lande 2016). Although animal biologists have often focused on the negative effects of inbreeding on offspring fitness, inbreeding provides reproductive assurance when mates are limiting and has inclusive fitness benefits that can counter and even outweigh the consequences of inbreeding depression (Fisher 1941; Lande and Schemske 1985).

These observations, along with renewed recognition of the generality of the forces that drive the evolution of inbreeding and outbreeding in all taxa (Szulkin et al. 2013; Lehtonen and Kokko 2015; Duthie and Reed 2016), set the stage for examining the role of inbreeding in shaping the evolution of diverse traits in marine invertebrates. Experience from the study of the evolution of plant-mating systems has much to offer in guiding such an effort. For example, plant biologists have described a selfing syndrome in which traits including small stature, rapid life cycle, more limited dispersal, and reduced investment in male gametes and their export are jointly associated with the evolution of higher levels of self-fertilization. A parallel syndrome has been recognized in hermaphroditic animals such as Caenorhabditis nematodes and Biomphalaria aquatic snails (Cutter 2019), and description of a corresponding inbreeding syndrome for marine invertebrates with separate sexes would help identify both individual traits and evolutionary transitions most likely to have been shaped by mating system evolution.

An obstacle to identifying mating system syndromes in marine invertebrates is that the vast majority of data on inbreeding in marine systems comes from population genetic estimates such as FIS. The extent to which these estimates actually reflect degrees of coancestry among mates is not known, and it has been suggested that sampling artifacts swamp the ability of these metrics to detect meaningful differences in inbreeding (Addison and Hart 2005; Waples 2015, 2018). More direct estimates of the degree to which individuals inbreed based on arrays of progeny genotypes, an approach that is common in the study of plant-mating system evolution, offer a valuable complement to population-level estimates. Convincing quantitative estimates of inbreeding for marine invertebrates would open the door to fundamental questions about how inbreeding contributes to the evolution of traits such as lifespan, investment in male gametes, and reproductive timing, and to the major evolutionary transitions between alternate modes of sexual expression, larval development, and mate choice.

Here, we aim to facilitate access to and application of theory, empirical results, and technical and experimental approaches from decades of the study of plant-mating systems for marine invertebrate ecologists interested in exploring parallels in their systems. We offer a brief primer on the major evolutionary forces influencing inbreeding and outbreeding, describe common techniques used to quantify inbreeding and its consequences, and provide guidance for relating botanical terms to their zoological counterparts. As examples, we also describe how concepts and findings developed in the plant literature could illuminate the evolution of mating system syndromes and the evolution of internal fertilization from broadcast spawning.

General theory for the evolution of inbreeding and outbreeding

Two opposing genetic forces, inbreeding depression and the automatic transmission advantage to inbreeding, influence the evolutionary advantages and disadvantages of inbreeding in all sexually reproducing taxa. Inbreeding depression, the lower fitness of inbred relative to outbred offspring, can be substantial and is well documented in a variety of plants (Husband and Schemske 1996; Winn et al. 2011) and animals (Crnokrak and Barrett 2002; Keller and Waller 2002). The automatic transmission benefit accrues to any level of inbreeding, but is greatest and most obvious in the case of an allele for self-fertilization arising in an otherwise completely outcrossing population of hermaphrodites (Fisher 1941). Such an allele has an automatic 50% advantage because an individual that both self-fertilizes and outcrosses transmits three copies of the selfing allele to its offspring (two as both dam and sire to selfed offspring and one as sire to outcrossed offspring), while an outcrosser can only transmit two copies of the alternative allele (as dam or as sire to outcrossed offspring). This automatic transmission bias also arises for an allele that promotes inbreeding in an otherwise outbreeding population of organisms that are not capable of self-fertilization, because mating among relatives increases the probability that multiple copies of the allele are transmitted to the next generation (Kokko and Ots 2006). The magnitude of the advantage declines as the degree of relatedness between mates decreases (Bengtsson 1978), but it can still favor the evolution of biparental inbreeding over outbreeding (Kokko and Ots 2006; Parker 2006).

Theory derived for hermaphroditic organisms shows that the balance between inbreeding depression and automatic transmission generates an unstable equilibrium level of inbreeding depression above which selection favors ever increasing outbreeding, and below which selection leads inexorably toward greater inbreeding (Lande and Schemske 1985; reviewed by Goodwillie et al. 2005). This instability is driven by the purging of deleterious recessive mutations and a consequent reduction in the magnitude of inbreeding depression in inbred populations (Charlesworth and Charlesworth 1999). Distributions of the rates of outcrossing estimated for large numbers of hermaphroditic plants (Goodwillie et al. 2005) and animals (Jarne and Auld 2006) have modes near the extremes of complete selfing and complete outcrossing, supporting the general importance of automatic selection, purging, and inbreeding depression in the evolution of inbreeding mating systems. Since past inbreeding influences current genetic load, selection against inbred offspring should be weaker in species and populations with a history of inbreeding. Inbreeding depression does tend to be weaker in plant populations with high selfing rates (Husband and Schemske 1996; Winn et al. 2011), but the effects of mating system history on the costs of inbreeding have rarely been tested in marine invertebrates (but see Cohen 1990, 1996; Palmer and Edmands 2000; Hughes et al. 2009; Phillippi and Yund 2017). Newer theory incorporates some of the complexities introduced by variation in the degree of inbreeding in animals with separate sexes (Kokko and Ots 2006; Szulkin et al. 2013; Duthie and Reid 2016), but inbreeding rates for these animals have not yet been comprehensively surveyed.

Another general force that can favor inbreeding under circumstances that would otherwise select for outbreeding is the need for reproductive assurance under conditions of mate or gamete limitation. When suitable mates or gametes are scarce, individual fitness may be increased by accepting a related mate over no mate at all. Pollen limitation of seed production in plants (Burd 1994; Knight et al. 2005) and sperm limitation of fertilization in animals (Levitan and Petersen 1995; Yund 2000) could select against mechanisms that prevent inbreeding to facilitate reproductive assurance. There is extensive documentation of pollen limitation and its association with plant traits and environments (reviewed by Burd 1994; Knight et al. 2005). Similarly, the likelihood of sperm limitation affecting fertilization success in marine animals has been linked to whether eggs are fertilized internally or externally (spermcasting and copulating versus broadcasting), adult-spawning behavior, the quantity and quality of available gametes, the spatial distribution and abundance of spawning individuals, and patterns of water flow (Levitan 1998; Pemberton et al. 2003). Selfing is known to increase in plants when pollen and pollinators are rare (Kalisz and Vogler 2003), but the association between sperm limitation and self-fertilization or biparental inbreeding has rarely been explored in marine invertebrates (but see Manriquez and Castilla 2005; Boissin et al. 2008).

Reproductive assurance can come at the cost of gamete discounting, which occurs when inbred mating usurps potential outbreeding. For example, there is an opportunity cost to an egg accepting the sperm of a relative if it suffers from reduced fitness and could have been fertilized by an unrelated individual. This would constitute egg discounting in animals or ovule discounting in plants. For plants and for animals that invest additional resources in offspring after fertilization, seed or offspring discounting can also occur. Sperm or pollen discounting can arise similarly because male gametes used in inbreeding are precluded from outbreeding with available female gametes. In iteroparous organisms, accepting a related mate during one reproductive bout can exact a gamete discounting cost if this reduces resources available for outbreeding in a future bout when the gametes of unrelated individuals are not limiting. Theory and data on how gamete discounting shapes the likelihood of inbreeding come nearly exclusively from plants (Kohn and Barrett 1994), but this cost to inbreeding may also play an important role in marine invertebrate mating systems.

The direct costs and benefits of inbreeding to individuals can be modified by kin selection and evolutionary conflict within families (Kamel et al. 2010a). For example, inclusive fitness gained through kin recognition can be substantial and has been linked to settlement patterns that promote spatial associations of relatives in bryozoans and ascidians (Keough 1984; Grosberg and Quinn 1986). Moreover, the intensity of conflict within families, such as competition for maternal resources or sibling cannibalism, depends on the degree of relatedness among family members (Kamel et al. 2010b). Thus, kin selection and family conflict likely contribute to both the benefits of inbreeding and the evolution of mechanisms to avoid it when the costs outweigh the benefits. These forces should be part of the framework for understanding marine invertebrate mating systems.

The major forces influencing the evolutionary costs and benefits of inbreeding and outbreeding are expected to be the same for plants and animals. Incorporating automatic transmission, and the effects of inbreeding history, reproductive assurance, and gamete discounting into thinking about the factors contributing to mating system variation in marine invertebrates could offer novel insight into persistent puzzles in this group such as repeated transitions from external to internal fertilization and associations among reproductive, dispersal, and ecological traits. Approaches and techniques that are common in the plant literature but rarely applied in animals could help quantify how much inbreeding occurs in marine invertebrates and determine why inbreeding or outbreeding occurs under particular conditions.

Quantifying inbreeding

A first step to understanding the role of inbreeding in the evolution of marine invertebrates is to obtain reliable measures of inbreeding in natural populations. Common measures of inbreeding either quantify how inbred a set of individuals, populations, and species are, or they estimate how much individuals engage in inbreeding. Population-genetic estimates of FIS, effective population size (Ne), runs of homozygosity (ROH), and linkage disequilibrium fall into the former category, and measures of the selfing rate, biparental inbreeding, and the degree of relatedness among mates into the latter. Importantly, these categories estimate distinct phenomena, and both have inherent limitations. For example, mating system estimates that rely on population genetics reflect the cumulative effects of inbreeding and subsequent selection. Thus, inbreeding can be considerable, but because inbreeding depression eliminates inbred offspring prior to the life stage at which sampling occurs, it can go undetected with a single population-genetic estimate. Alternatively, measures of the degree to which individuals inbreed do not necessarily reflect how inbred or outbred the population in question may be.

The majority of studies that evaluate inbreeding in marine invertebrates compare population-level patterns of observed heterozygosity to those expected under Hardy–Weinberg equilibrium to calculate the inbreeding coefficient, FIS. However, evidence for inbreeding from FIS is often met with skepticism because non-amplifying genetic variants (null alleles; Waples 2018) and undetected population structure (the Wahlund effect; Waples 2015) can artificially inflate homozygosity (Addison and Hart 2005). Approaches that combine multiple techniques to evaluate both how inbred individuals are and how much they inbreed could help circumvent common sources of error and move the study of marine invertebrate mating systems forward (see Sherman 2008; Carlon and Lippe 2011; Ledoux et al. 2020).

Studies of plant-mating systems typically quantify the degree to which an individual inbreeds based on the frequencies of genotypes in a set of its offspring (a progeny array). Methods developed by Ritland and Jain (1981) and Ritland (2002) use the genotypes of offspring (or a parent and its offspring) to estimate the fraction of progeny produced from self-fertilization or random outcrossing and can account for loci harboring null alleles. Numerous measures from progeny arrays suggest that selfing rates vary widely both within and among species of seed plants (Whitehead et al. 2018), and are driven by ecological and evolutionary factors such as mate availability and inbreeding history (Goodwillie et al. 2005). In contrast, there are relatively few estimates of selfing rates or biparental inbreeding based on progeny arrays under natural conditions for marine invertebrates (but see Brazeau et al. 1998; Boissin et al. 2008; Sherman 2008; Carlon and Lippe 2011; Ledoux et al. 2020). Selfing rates can be estimated indirectly from the genotype frequencies of populations rather than with progeny arrays (see Cohen 1990; Jarne and Auld 2006), but these estimates of inbreeding are likely to have limitations similar to other population-genetic surveys.

Studies of inbreeding in marine invertebrates often test the compatibility of related or self-gametes in the laboratory (Heyward and Babcock 1986; Cohen 1996; Johnson 2010; Barbosa et al. 2012; Kosman et al. 2017; Burgess et al. 2019), rather than the frequency and magnitude with which inbreeding occurs. These studies typically find a low but variable capacity for self-fertilization or biparental inbreeding (Bishop et al. 1996; Knowlton et al. 1997; Levitan et al. 2004; Fogarty et al. 2012), which promotes the view that inbreeding is rare in these species. However, the ability to inbreed is likely an evolutionarily labile trait that varies among individuals and populations (Grosberg and Hart 2000). Thus, measures of compatibility are useful for evaluating the molecular basis of self-recognition (Sawada et al. 2020), but do not indicate if or how much inbreeding varies among populations and species and consequently offer little insight into conditions favoring inbred or outbred mating systems.

Progeny arrays can also be used to estimate biparental inbreeding (Ritland 2002), which is expected to facilitate the evolution of selfing by gradually increasing homozygosity and purging inbreeding depression (Uyenoyama 1986; Porcher and Lande 2016). However, theory and data on the causes and extent to which biparental inbreeding varies within and among species is relatively slim compared with self-fertilization. Given the diversity of ways that syngamy is achieved in marine invertebrates, this group could provide powerful tests of why differences in biparental inbreeding occur outside the context of self-fertilization. The use of progeny arrays to estimate inbreeding in species that reproduce by spermcasting or copulation is straightforward because internally developing offspring are typically easy to obtain, but estimating natural selfing or biparental inbreeding rates in broadcast spawning species would likely require netting fertilized embryos shortly after mass spawning events (as in Levitan et al. 2004). Analysis of progeny arrays could serve as a standardized method for quantifying inbreeding in marine invertebrates, permitting broad analyses that evaluate its associations with reproductive traits and environmental conditions, as it has in plants.

Combining parentage assignment with measures of relatedness among mates can also quantify the strength of biparental inbreeding (see Plough et al. 2014). Estimating the degree of relatedness among pairs of mates could indicate whether individuals avoid, prefer, or tolerate inbreeding and how factors already known to contribute to non-random mating such as proximity, individual size, and spawning synchrony interact to influence reproductive success. A limitation of this approach is that values of relatedness are not standardized across studies (Weir et al. 2006). Relatedness estimates reflect the probability that individuals share alleles that are identical by descent (Lynch and Ritland 1999; Wang 2011), but differences in the number of genetic markers analyzed, their polymorphism, and the demographic history of the species and population in question can bias estimates relative to theoretical expectations. However, when relatedness estimates are calibrated with individuals of known coancestry, such as full and half siblings, or parents and offspring, numeric values of relatedness can be interpreted as quantitative measures of coancestry.

Zoological equivalents of plant-specific concepts and terminology

Plants have evolved numerous adaptations to promote or prevent inbreeding, which feature strongly in the literature (reviewed in Harder and Barrett 2006). Identifying and documenting corresponding adaptations that promote or prevent inbreeding in animals is a critical step towards understanding how these aspects of mating can shape patterns of trait variation and their evolutionary consequences in marine invertebrates. However, despite similarities in how plants and animals distribute sexual functions in time and space and how this creates different pathways for inbreeding that have distinct costs and benefits, much of the language in the current literature is specific to plants. Here, we offer definitions of botanical terms and concepts common in the literature on mating system evolution, along with their approximate counterparts in marine invertebrate biology (Box 1; 2), while highlighting conceptual similarities in mating system evolution in these two groups.

For both animals and plants, spatial and temporal proximity of male gamete release and female gamete receptivity are required for inbreeding to occur. Therefore, traits that influence relative timing and proximity provide information about the likelihood that inbreeding occurs and/or is favored. In addition, the costs and benefits of inbreeding can depend on whether the uniting male and female gametes were produced in the same structural unit and the timing of inbreeding relative to the opportunity for outbreeding.

Like plants, individual animals can produce either only sperm, only eggs, or both, either sequentially or simultaneously. The basic unit of Angiosperm reproduction is the flower, which can contain either structures that produce male gametes, structures that produce female gametes, or both may be contained within a single flower. Due to their modular construction, most individual plants bear multiple flowers, which necessitates a set of terms to describe the alternative possible spatial distributions of male and female organs within and among individual flowers and organisms (Box 1; Fig. 1). A large majority of seed plants are hermaphroditic (Renner and Ricklefs 1995), and while hermaphroditism is common in some marine invertebrate taxa (Carlon 1999; Jarne and Auld 2006), the distribution of species with separate sexes (dioecy or gonochory) or hermaphroditism is not as well characterized for marine invertebrates. More complicated sex expression also occurs in both groups (e.g., gynodioecy, in which there are males, females, and hermaphrodites), and these less common forms are thought to be intermediates in evolutionary transitions between hermaphroditism and dioecy in plants and animals (Charlesworth and Charlesworth 1978; Weeks 2012).

Fig. 1
figure 1

Spatial arrangement of male and female sex organs in plants and marine invertebrates

Full size image

Additional terms describe the temporal synchrony of male and female function (Box 1), which can influence relative timing of inbreeding and outbreeding. The relative timing is important because the costs and benefits of inbreeding can depend on whether it occurs prior to, simultaneous with, or after opportunities for outbreeding (Lloyd 1992). For example, inbreeding that occurs after all opportunities for outbreeding have passed does not incur a gamete discounting cost.

Unlike Angiosperms, all of which produce gametes within anthers and pistils, the gamete-producing structures of marine invertebrates vary from an informal cluster of cells that arise from somatic tissue (e.g., sponges and cnidarians) to specialized organ systems (e.g., ascidians). These differences contribute to determining the magnitude of temporal and spatial separation of male and female function, and therefore, the potential for marine invertebrates to inbreed and the consequences when they do. Alternate spatial arrangements of male and female reproductive structures also create distinct pathways for male gamete movement that result in self-fertilization, and these are recognized to have different consequences for the cost and reliability of inbreeding via selfing (Box 2; Fig. 2). For example, self-fertilization involving gametes produced by different flowers of the same hermaphroditic plant (geitonogamy) is often not adaptive because it would be more efficient to produce both gametes in a single flower (autogamy; Eckert 2000).

Fig. 2
figure 2

Parallel pathways of fertilization within and between modules and genotypes for a coral and an Angiosperm. Autogamy or intra-polyp selfing occurs when the sperm of a hermaphroditic individual fertilizes an egg produced in the same flower or polyp. Geitonogamy or intra-colony selfing occurs when the sperm of a hermaphroditic individual is transferred to the egg of a different flower or polyp produced by the same genetic individual. Xenogamy or inter-colony outcrossing occurs when sperm is transferred to an egg produced by the flower or polyp of a genetically distinct individual. Biparental inbreeding occurs through Xenogamy between two related individuals

Full size image

Applying theory on inbreeding and outbreeding to marine invertebrates

Mating system syndromes

In plants, two alternative evolutionary states referred to as the “selfing syndrome” and the “biotic pollination syndrome” are recognized as the product of selection acting on multiple affiliated traits to promote or limit inbreeding. High selfing rates are associated with reduced floral displays, limited seed dispersal, and shorter lifespans, whereas predominant outcrossing is associated with increased pollen production, greater seed export, and longer lifespans (Barret et al. 1996; Hamrick and Godt 1996; Goodwillie et al. 2010). These alternative syndromes have repercussions for population and species level attributes including gene flow, genetic load, range size, and niche breadth that could help generate hypotheses linking mating systems with the ecology and evolution of marine invertebrates.

Populations and species of highly selfing plants have smaller and fewer flowers, lower pollen to ovule ratios, and reduced separation of male and female function in space (herkogamy) and time (dichogamy) (Box 1) compared with their outcrossing relatives. These floral differences are not a degeneration of traits as a result of inbreeding depression, but rather adaptions to the conditions favoring inbreeding (Shimizu and Tsuchimatsu 2015). Selection for increased self-fertilization can lead to the evolution of reduced allocation to pollen-producing structures, restricting the effective export of pollen. Similarly, traits of marine invertebrates associated with the separation of male and female functions in hermaphroditic species, and male reproduction in general, may evolve in response to selection to promote or avoid inbreeding. Greater FIS in marine invertebrates that simultaneously express male and female functions relative to other forms of hermaphroditism support this notion (Olsen et al. 2020), but it is unclear whether selection directly favors inbreeding in these cases or if increased inbreeding is a byproduct of selection acting on other factors. Moreover, testis size, sperm production, velocity, and longevity vary considerably within and among species of marine invertebrates and have well characterized repercussions for male competition and sperm limitation (Yund et al. 1997; Levitan 2000; Johnson and Yund 2004). In addition, variation in sperm traits may in part be driven by the costs and benefits of avoiding inbreeding when relatives are in close proximity. Studies addressing how sperm traits interact with the relatedness, proximity, and density of mates to affect reproductive success are warranted to tease apart the contributions of sexual selection and inbreeding to male variation. The greater overlap in male and female functions and reduced pollen export in inbreeding plants suggest the following predictions: (1) selection for inbreeding leads to an evolutionary increase in the temporal and/or spatial overlap in male and female functions in hermaphroditic species of marine invertebrates and (2) selection for inbreeding leads to an evolutionary reduction in male reproductive structures, sperm production, sperm longevity, and the effective export of male gametic material in marine invertebrates. These hypotheses are particularly relevant for sessile and hermaphroditic species and tests could provide insight into the coevolution of reproductive traits in terrestrial plants and marine animals under conditions where inbreeding may be adaptive.

While the shift from predominant outcrossing to a selfing mating system occurs frequently in plants with annual lifespans (Barrett 2002a), longer lived species are far less likely to evolve high selfing rates (Barrett et al. 1996). The association between mating system and lifespan is likely tied to strong selection for reproductive assurance in semelparous organisms and the ineffectiveness of purging to remove late-acting inbreeding depression (Husband and Schemske 1996; Lesaffre and Billiard 2020). The relationship between rates of self-fertilization and longevity in plants suggests the prediction of a negative association between lifespan and inbreeding in marine invertebrates. Abundant mutations and broad self-incompatibility in long-lived reef corals provides circumstantial support for this prediction (Carlon 1999; Olsen et al. 2019), but comparisons of inbreeding rates and stage-specific inbreeding depression among closely related species with different lifespans would comprise stronger tests. For example, within the seastar family Asterinidae, there is considerable variation among species in individual size, sexual configuration, larval form, and likely lifespan (Hart et al. 1997), making this an intriguing group for evaluating how inbreeding may vary with different traits.

The correlated evolution of mating systems, seed dispersal, and colonizing-ability, a longstanding focus of botanical research (reviewed by Pannell 2015), is paralleled by relationships between inbreeding, larval dispersal, invasion success, and competitive ability in marine invertebrates (Knowlton and Jackson 1993; Addison and Hart 2005; Dupont et al. 2007; Olsen et al. 2020). Since self-fertilization permits reproductive success under conditions where an individual is isolated, inbreeding can be adaptive at the periphery of a species range or in locations where mate availability is inconsistent. Climate-induced range expansion and other anthropogenic disturbances that alter the demography of plants and marine invertebrates can establish populations with increased inbreeding and reduced effective size (Pujol et al. 2009; Dawson et al. 2010). However, the repercussions of genetic bottlenecks are likely dependent on the inbreeding history of the species or population in question, and may have catastrophic effects on the fitness of some groups and little impact on others (Edmands 2007). For example, Knowlton (2001) suggests that on heavily disturbed coral reefs comparatively “weedy” corals (e.g., Favia fragum) characterized as physically small, short-lived, spermcasting, and relatively inbred, are replacing larger, longer lived, and historically dominant reef-builders (e.g., Orbicella) which outbreed via broadcast spawning and have the potential for long distance dispersal. These observations are qualitatively consistent with the only formal review of selfing specific to marine invertebrates, which consists of 8 studies evaluating 26 species of reef-building corals (Carlon 1999). Expanding the use of progeny arrays would help confirm connections between inbreeding, dispersal, lifespan, and the ecology of marine invertebrates.

The theoretical and experimental evidence in both plants and marine invertebrates suggest the predictions that (1) inbreeding, outbreeding, and larval dispersal in marine invertebrates are connected through correlated evolution, and this leads to the evolution of alternate associations of outbreeding with high dispersal potential and inbreeding with low dispersal potential and (2) the ability to inbreed promotes colonization success, wide niche breadths and “weedy” lifestyles. Intraspecific comparisons of the magnitude of inbreeding in species with multiple larval modes (poecilogony), such as sacoglossan gastropods (e.g., Alderia sp.) and spionid polychaetes (e.g., Streblospio sp.) would test the former prediction, and contrasting the mating systems of closely related species with different invasion histories (see Dupont et al. 2007) and niche breadths could test the latter.

Inbreeding and the evolution of internal fertilization

Examining the diversity of reproductive traits in marine invertebrates through the lens of theory developed for plants can provide a novel perspective on the processes that dictate the speed and direction of life cycle trait evolution. For example, the division between species that brood versus broadcast eggs is a longstanding evolutionary puzzle. The evolutionary transition from broadcast spawning and external fertilization to spermcast mating and internal fertilization is thought to be common and the forces promoting this transit have been viewed primarily in the light of adult size, asexuality, sexual selection, sperm limitation, and life history constraints (Strathmann and Strathmann 1982; Lively and Johnson 1994; McHugh and Rouse 1998; Parker 2014). Spermcast mating tends to be associated with the production of short dispersing larvae, which has led to an expectation that brooding taxa are likely to engage in inbreeding (Knowlton and Jackson 1993). However, the ability to concentrate sperm through active filtering and to retain mature eggs until sperm is available suggest that brooding species are less sperm-limited and have greater control over paternity than their broadcasting counterparts (Bishop et al. 1996; Pemberton et al. 2003). To assure reproduction, broadcast eggs are expected to be less selective about accepting the sperm of a relative, because the risk of fertilization failure is higher. Here, the costs and benefits of inbreeding for a given individual are likely to reflect the balance between dispersal traits that govern the likelihood of settling near a relative and reproductive assurance garnered through inbreeding. It has been suggested that the evolution of internal fertilization is in part driven by selection to increase female control over paternity (Levitan 1998; Parker 2014), and we propose that the ability to promote or avoid inbreeding may be an important element of this adaptation. Comparatively low FIS values in species that reproduce by copulation despite the absence of a planktonic dispersal stage generally support this notion (Addison and Hart 2005; Olsen et al. 2020), but empirical data on reproductive assurance, mate choice, inbreeding depression, and the prevalence of inbreeding and outbreeding across taxa where spermcasting and copulation have recently evolved from broadcast spawning are needed to evaluate the relative merits of this hypothesis.

Conclusion

Drawing from the theory, techniques, and literature of plant-mating systems could help accelerate progress towards identifying and explaining variation in inbreeding and outbreeding in marine invertebrates. Our current understanding of the ecological and evolutionary factors influencing the prevalence of inbreeding is largely limited to population genetic studies and laboratory-based tests of self-compatibility. Directly assessing individual levels of inbreeding using progeny arrays and taking into consideration the automatic transmission advantage, and the effects of inbreeding history, reproductive assurance, and gamete discounting could provide a complementary perspective on the correlated evolution of reproductive, dispersal, and ecological traits and the evolution of internal fertilization. Moreover, the diversity of traits associated with syngamy and development found across marine invertebrates offer opportunities for expanding the theory of inbreeding and outbreeding to accommodate a broader range of phenomena.

Box 1: Terms describing the spatial and temporal distribution of male and female functions

Spatial and temporal separation of the production and release of male and female gametes in plants and animals can both promote outcrossing and reduce interference between male and female functions (reviewed by Barrett 2002b). The term dioecy describes the condition of having separate sexes, in which each individual produces either only male or only female gametes such that self-fertilization is not possible, and direct sexual interference is minimized. This condition is relatively rare among seed plants (~ 6%; Vamosi and Vamosi 2004) in contrast to many animal groups in which separate sexes, a condition that has been called either dioecy or gonochorism, is common (~ 94%; Jarne and Auld 2006).

Individuals of the majority of plant taxa are hermaphroditic, and botanists distinguish two major groups of hermaphrodites based on the spatial arrangement of male and female sexual organs (Fig. 1). The term monoecy designates a form of hermaphroditism in which single flowers produce either only male or only female gametes, but individual plants bear flowers of both sexes. In contrast, hermaphrodites that produce male and female organs within the same flower are said to have perfect flowers. For animals, the term monoecy is equivalent to hermaphroditism without any distinction of degree of spatial separation of male and female functions. In hermaphroditic, colonial animals, zooids are typically akin to perfect flowers, producing both eggs and sperm. However, examples where male and female functions are physically separated are found in Scleractinian corals (e.g., sex-segregated polyps in some populations of Astroides calycularis), Hydrozoans (e.g., medusae in Obelia), Bryozoans (e.g., Celleporella hyalina), Ascidians (e.g., mixed male and hermaphrodite zooids arising from cycles of sequential gonad development Botryllus schlosseri), among other groups.

Additional terms describe patterns of sex expression that are less common in either plants or animals and that are generally thought to reflect evolutionary transitions between hermaphroditism and dioecy in both groups (Lloyd 1975; Charlesworth and Charlesworth 1978; Weeks 2012). Gynodioecy refers to species that comprise both female and hermaphrodite individuals (e.g., the anemone Epiactis prolifera), and androdioecy describes species such as stalked and acorn barnacles that include both males and hermaphrodites. Inbreeding depression and reproductive assurance are proposed to be dominant forces driving evolutionary transitions between the extremes of dioecy and hermaphroditism in both plants (Lloyd 1975; Charlesworth and Charlesworth 1978; Pannell 2002) and animals (reviewed by Weeks 2012).

When there is spatial separation of female and male gamete-producing structures within a perfect flower or a hermaphroditic zooid, the degree of separation can determine the likelihood of inbreeding by self-fertilization. Spatial separation of male and female functions within a flower is referred to as herkogamy. Although there is no equivalent term for animals, the magnitude of spatial separation between gamete-producing regions varies among taxa. For example, the lack of formal reproductive organs in sponges or corals can lead to sperm and eggs being produced and released within microns of each other, whereas the sophisticated organ systems in a tunicate or bryozoan may include a physical and spatial barrier between ovaries and testes.

For both plants and animals, the temporal separation of male function (gamete release) and female function (egg release or receptivity to fertilization) can have consequences for the opportunity for inbreeding similar to those of spatial separation. The general term dichogamy refers to temporal separation of male and female functions in plants and the corresponding term for animals is sequential hermaphroditism. In both plants and animals, protogyny is used to designate female function occurring prior to male, and protandry the reverse.

Box 2: Terms describing patterns of male gamete movement within and among flowers and individuals

Pollen transfer between the flowers of genetically distinct individuals is called xenogamy and fertilization resulting from such movement is called allogamy or outcrossing. In both plants and animals, the term outcrossing encompasses biparental inbreeding as well as outcrossing with unrelated mates.

Alternate spatial arrangements of male and female reproductive structures create distinct pathways for male gamete movement that result in self-fertilization in plants, and these are recognized to have different consequences for the cost and reliability of inbreeding via selfing (Fig. 2). Self-fertilization can be achieved by the transfer of pollen within a single hermaphroditic flower (autogamy) or by the transfer of pollen from one flower to another produced by the same hermaphroditic individual (geitonogamy). Both autonomous autogamy (selfing that occurs without the aid of a pollen vector) and geitonogamy result in self-pollination, but geitonogamy is more expensive because it requires the construction of multiple reproductive structures and is less reliable because it requires an external vector to transfer pollen. For a species capable of autonomous selfing, any resources invested in attracting and rewarding a vector are effectively wasted when geitonogamy occurs. For hermaphroditic marine invertebrates, inbreeding that results from fertilization of eggs by sperm produced by different modules of the same individual (e.g., different zooids of the same colony) would be analogous to geitonogamy, and fertilization of eggs by sperm produced by the same module (zooid or polyp) would correspond to autogamy. Although marine invertebrates are not known to use animal vectors to transport their gametes (but see van Tussenbroek et al. 2012), any resources required to take advantage of gamete transport such as long dispersing sperm are likewise wasted when fertilization occurs as a passive consequence of close spatial and temporal proximity of male and female gamete production.