Origin of an evolutionary novelty: the worker phenotype of eusocial wasps

Complex problems in evolutionary biology can be approached in two ways, top down using theoretical constructs and bottom up using empirical studies . Theoretical concepts predominate evolutionary interpretations of eusociality in a literature that is small relative to an enormous literature of natural history and basic research that is not synthesized into a conceptual whole. Here, I draw insights from this literature to show how paper wasps’ allomaternal non-reproductive worker phenotype originates in every colony cycle via confluence of multiple factors of paper wasp biology. These include behavior, development, nutrient dynamics, indirect genetic effects, sex ratio, and demography. A novel perspective on the colony cycle, based on individuals’ reproductive physiology, serves as context to examine of each of these. It will be shown that the allomaternal non-reproductive worker phenotype does not require relatedness among colony members to originate. Allomaternal care of non-relatives is frequent and can occur in at least twelve contexts. Life histories of living species as they will be presented here show that relatedness among colony members is not the target of selection in simple eusociality. However, the novel allomaternal non-reproductive worker phenotype had to be present at the ancestral origins of complex eusociality in which relatedness among colony members is essential.


Introduction
A fundamental issue that many evolutionary biologists have wrestled with, often termed the selection paradox, is that selection can only sort among existing variation but cannot create novel phenotypes (Pigliucci 2008). Creativity in evolution must come from elsewhere. Eusociality is perhaps the best-known example of a life history type that calls for evolutionary explanation, and presence of a non-reproducing worker phenotype is the essence of eusociality. The purpose of this review is to use empirical studies to unravel and understand factors foundational to the origin of the allomaternal non-reproducing worker phenotype of paper wasps. The central premise is that empirical factors alone can frame an hypothesis for that origin. The hypothesis asserts that the phenotype originates from a coming together of multiple aspects of paper wasp biology: nest architecture, larval development, maternal behavior, sign stimuli that release allomaternal care, energetic costs of allomaternal care, dynamics of relationships among adults, and demography. Nourishment flow among colony members sets the context for indirect genetic effects that shape the phenotype of every colony member. Relatedness between workers and the future reproductive larvae they care for plays no role.
The context that will frame the hypothesis is the annual colony cycle of paper wasps. Van Gestel and Tarnita (2017) offer a perspective on the origin of biological construction that, although focused on multicellularity, includes paper wasps in outlining a bottom-up dynamical account of how, starting from a solitary ancestor, groups originate and subsequently evolve the organizing principles that qualify them as new units. Their perspective stresses the paramount importance of the life cycle and highlights the crucial role of ecology and development of a solitary ancestor. They argue that unraveling the evolutionary trajectory of hierarchical transitions (e.g., solitary to eusocial) is only possible by evaluating groups in the context of their life cycle. Batra (1966) coined the term "eusocial" to characterize only bees in subfamily Halictinae with subsocial (mother/ daughter) colonies and no morphological differentiation among colony members: overlap of generations, cooperative brood care, and division of reproductive labor (i.e., reproductives and non-reproducing workers). In its broader application to wasps, bees, and ants, the term eusocial has been divided into two categories, with terminology that corresponds to those categories but differs among authors: eusocial (Batra 1966) and hypersocial (Batra 1995); primitive and advanced (Wilson 1971); behavioral and morphological (Kukuk 1994); simple and complex (Bourke 1999).
Paper wasps (Hymenoptera, Vespidae, five genera of Polistinae) have simple eusociality as characterized by Bourke (1999, p. 245): few or no morphological differences between reproductive individuals and workers, no physical caste polymorphism among the workers, and relatively simple nests and communication systems. In addition, individuals have reproductive flexibility, although there are boundaries to its expression. The word polistes comes from classical Greek and means founder of a city. Although extant colonies are sometimes initiated by multiple foundresses, the ancestral state of paper wasp eusociality is solitary founding. Paper wasp simple eusociality per se begins when the first female offspring emerge from pupation and engage in allomaternal care of nestmate larvae. The single foundress, which has now become queen, and her matrifilial colony can be easily observed and experimentally manipulated in both field and laboratory. Importantly, paper wasps have an annual life cycle, which facilitates experiments and observations focused on single elements of that life cycle.
The genus Polistes has become a model system (Reeve 1991;Turillazzi and West-Eberhard 1996;Jandt et al. 2013) such that, together with other genera, especially Mischocyttarus (e.g., Jeanne 1972;Hunt 1988;Fiocca et al. 2020;Cervo and Dani 1996) and Ropalidia (e.g., Gadagkar 1991Gadagkar , 2001, a substantial body of knowledge makes paper wasps ideally suited to seek to understand the relevant ecological factors foundational to the origin of an allomaternal non-reproducing worker as a novel phenotype. What follows here is a topical review in which each topic constitutes a brick in that foundation. It begins with an overview of Polistes life history placed in the context of its annual colony cycle, including a novel presentation based on reproductive physiology of individual females rather than state of the colony at a particular point in the cycle. Citations of empirical findings support each topic. Single elements of Polistes' biology, when taken collectively, show how the allomaternal non-reproducing worker phenotype originates, even in the absence of intra-colony relatedness. A small number of theoretical models support the proposition that relatedness is not required for the novel phenotype to originate. The discussion begins by summarizing the body of the paper and continues with descriptions of how ideas drawn from the review can be placed into broader contexts.

Prerequisites
The matrifilial generational overlap in paper wasps was based on convergence of four aspects: nest architecture, larval provisioning behavior, larval development, and chemical communication.
(1) The solitary mud-dauber wasp Sceliphron assimile, which is unrelated to paper wasps, constructs a nest cell, stocks it with spiders as food for the larva that will develop within, lays an egg, closes the cell, then initiates construction of another nest cell that need not be attached to the one just closed. In a demographic study, overall larval mortality was high, with 60% of deaths at the pupal stage caused by parasitoids or cleptoparasites (Hunt 1993). Other solitary wasps suffer similar losses (Cowan 1991). (2) A paper wasp foundress initiates new nest cells one at a time and attaches them to previously initiated cells. A newly initiated nest cell is a shallow cup into which the foundress lays an egg. Lengthening of nest cells and initiation of new cells continue synchronously, leading to an ever-enlarging nest.
Nest cells are open, each containing a developmental stage from egg to fifth (last) instar larva. Paper wasps surmount the parasitoid problem by kneading prey items into an unrecognizable mass of tissue, thereby eliminating the possibility of introducing parasitoids into their larvae (Cowan 1991). Kneaded prey items would decay quickly, thus necessitating direct feeding to larvae. The foundress feeds larvae synchronously until each completes larval development, spins a silk cocoon, and encloses itself for metamorphosis. (3) Newly molted adults emerge serially, just as nest cells were initiated serially. Longitudinal folding of the forewings characterizes Vespidae (excluding Stenogastrinae, Euparagiinae, and most Masarinae) (Danforth and Michener 1988). The trait is so distinctive that Faltenwespen (fold wasp) is the German vernacular name for vespid wasps. However, when a paper wasp emerges from pupation in the tight confines of its nest cell, the wing cuticle is teneral (soft and unsclerotized: upper photo in right-hand column and adult phenotype photos in Fig. 3 of Hunt et al. 2007), and the longitudinal fold does not appear until 36 h post-emergence (Danforth and Michener 1988). In consequence, the wasp cannot fly for at least the first 36 h of adult life. (4) Newly emerged wasps lack the distinctive cuticular hydrocarbon (CHC) odor that they acquire from the nest paper (Singer and Espelie 1992) between two and three days following emergence (Panek et al. 2001) and that serves as an essential component of nestmate recognition and non-nestmate discrimination (Layton and Espelie 1995;Gamboa 2004;Mitra et al. 2014). Lacking a chemical identification label, newly emerged adults are not ejected from the colony by other adults, and as they acquire the colony odor they are accepted as colony members.

Life cycle
Paper wasp simple eusociality is established anew in every generation (Hunt 1991) (Fig. 1). A foundress is, de facto, a solitary wasp that initiates a nest and executes the complete module of maternal care behaviors. Independent founding is one criterion of paper wasp eusociality.
Female wasps, both solitary and social, in Hymenoptera's suborder Apocrita emerge from pupation with undeveloped (synovigenic) ovaries that require nourishment for oocyte development. All wasps are "wasp waisted" and cannot ingest solids, therefore adults are restricted to liquids as nourishment sources. Larvae are grub-like ( Fig. 3 of Hunt et al. 2007) and face no restriction, therefore they can feed on solids. Adult paper wasps feed on floral nectar (Domínguez et al. 1989;O'Donnell 1995;De Souza et al. 2008), extrafloral nectar (Beckmann and Stuckey 1981), hemolymph of prey items fed to their larvae (Hunt 1984) or extracted exclusively for self-nourishment (Hunt and Noonan 1979), and can derive additional nourishment from pollen . When larvae eclose from eggs and begin larval growth, the foundress brings prey (generally caterpillars) to feed them. She kneads the prey item and extracts hemolymph (insects' "blood") prior to feeding the solid mass to larvae. She then regurgitates the hemolymph as a "second course," although she retains a small portion that serves as a source of liquid nourishment (Hunt 1984). At times other than when feeding larvae, the foundress imbibes saliva that larvae copiously produce. Larvae are capable of gluconeogenesis (Ishay and Ikan 1968) and act as energy converters, ingesting proteinaceous caterpillar tissue and converting it into a nourishing liquid containing glucose (Hunt et al. 1987) and all 20 nutritive amino acids at a richness up to 50 times more than any floral nectar (Hunt et al. 1982). Initially, larvae may have secreted saliva to lubricate dry food morsels that resulted from extensive kneading and hemolymph extraction by the foundress (Hunt 1984), but it then evolved to its present-day copious production and nutritional richness as a food source that can forestall cannibalism by the foundress (Hunt 1988;Hunt, 1991).
Larvae grow and increase in number as a foundress' nest expands, increasing her foraging energy costs (Weiner et al. 2012) and leading to ovary regression and a pause in oviposition (Haggard and Gamboa 1980;Hunt and Dove 2002). Most or all early offspring are females physiologically primed for reproduction (Bohm 1972) and have the capacity to found nests and lay eggs (Strassmann 1981;Itô and Yamane 1985;Itô 1986;Mead et al. 1995). During the flightless period following emergence, they encounter nestmate larvae producing saliva that, by virtue of its nutrient richness, can enable ovarian development. Once capable of flight, easy access to the copiously produced nourishing liquid induces them to return at the nest to obtain it. As they do so, stimulus cues from nestmate larvae needing to be fed release the module of maternal behaviors, minus oviposition, performed heterochronically and directed allomaternally toward larvae that are not their own. Allomaternal care is a response to the same sign stimuli that are releasers for maternal care by foundresses.
When allomaternal care begins, the foundress (now queen) ceases to forage and nourishes herself on larval saliva and on prey hemolymph and nectar taken from foragers. Her ovaries recrudesce, and she resumes oviposition at a higher level than as a foundress and with a concomitant higher rate of increase in colony size (Hunt and Dove 2002). Whereas the foundress initially faced only the energetic costs of maternal care of tiny first instar larvae, newly emerged females face energetic costs of allomaternal care of an existing and ever-increasing number of large larvae like those that caused the foundress' ovaries to regress. Consequently, their synovigenic ovaries fail to fully develop, rendering them functionally non-reproductive.
Allomaternal workers care for nestmate larvae that will become adult females called gynes, which are the potential foundresses of the next generation ( Fig. 1). Males emerge from pupation synchronously with gynes but play no role in social life beyond consuming nourishment at the nest. Gynes emerge with undeveloped synovigenic ovaries, but they neither engage in allomaternal care nor show ovarian development. Instead, they emerge in reproductive diapause with substantial stores of lipids and storage proteins (Hunt et al. 2003;Toth et al. 2009) that enable them to pass an unfavorable season (cold, hot and dry) in behavioral quiescence. At the start of the next favorable season, gynes emerge from quiescence to become foundresses with little or no remaining stored nourishment (West Eberhard 1969) and synovigenic ovaries in a pre-reproductive state. The next colony cycle begins.
Simple eusociality implies that all reproductive options are open to every wasp, but they are not. The queen dominates her daughters (Pardi 1948). A further dominance hierarchy exists among workers (Pardi 1948;O'Donnell 1998a) in which high ranking individuals secure more nourishment than lower ranking nestmates (Markiewicz and O'Donnell 2001;Fiocca et al. 2020). Dominance is a recurrent theme in sociality (West-Eberhard 1981), and in the critical period of early adulthood in insects with simple 1 3 eusociality, dominance is likely to be an important environmental control that regulates behavior (van Doorn and Heringa 1986;Jandt et al. 2013;Smith et al. 2021). "[F]emales with elevated ovarian development, high juvenile hormone titers, and a trophic advantage due to social dominance (a set of consistently correlated traits) are able to maintain reproductive and social dominance over other females, who are consequently pushed to the opposite extremes in these A B traits" (West- Eberhard 2003, p. 301). Individuals low in the dominance hierarchy are the primary foragers, face high energy costs of foraging, never show ovary development, and therefore have de facto lifetime sterility as workers. With overlapping generations, collective brood care, and reproductive division of labor now present, all three definitional components of simple eusociality have been met. Alexander (1974) introduced the term "parental manipulation of progeny" to encapsulate his argument that offspring can be "treated by parents as parental investment and may not in fact be allowed to maximize their own reproduction or even to reproduce at all" (Alexander 1974, p. 337). Maternal manipulations of the first brood of paper wasp adults take place via both direct (aggression: West-Eberhard 1986; Tibbetts and Reeve 2000; dominance behaviors: Markiewicz and O'Donnell 2001;Keeping 2002;Tibbetts and Dale 2004) and indirect (visual: Tibbetts et al. 2010;pheromonal: Sledge et al. 2001) means. Also, Polistes fuscatus foundresses vibrate the nest by rapidly and forcefully beating their antennae on nest cell rims, a behavior called antennal drumming (Pratte and Jeanne 1984;Suryanarayanan et al. 2011a) that could be a mechanical stressor modulating developmental pathways thought to underlie diapause (Suryanarayanan et al. 2011b;Jandt et al. 2017), with stressed larvae during the gyne production phase of the colony cycle having characteristics of non-diapause workers.

Nourishment dynamics
Pioneering paper wasp researchers (Marchal 1897; Roubaud 1916) proposed that a significant component of worker/gyne differentiation in paper wasps occurs during larval development and is based on differences in nourishment. Those developmental differences, in turn, affect reproductive physiology of emerged wasps: future workers (fed by the foundress) receive less nourishment during development than larvae of future gynes (fed by multiple workers). Subsequent studies have shown that quantitative differences in larval nourishment do correspond to differences in development and reproductive physiology (Gadagkar et al. 1988(Gadagkar et al. , 1990Karsai and Hunt 2002;Keeping 2002;Hunt et al. 2003;Judd et al. 2015;Toth et al. 2009).
Nourishment flow threads a web of interactions among all colony members (Fig. 2). Foundress-reared larvae have a net gain as they grow, but the foundress exploits their capability as energy converters when she takes saliva from them. When allomaternal workers emerge, the queen feeds not only on larval saliva but also on hemolymph of prey taken from foragers and liquids surrendered by workers during dominance interactions. The queen thereby has a substantial net gain. Larval gynes and males develop without a nutritional loss. Dominant workers gain in food transfers with subdominants. Workers experience a nutritional loss at every food transfer, which reinforces their inability to develop their ovaries, enhances and sustains the queen's oviposition, enhances physiological characteristics that elevate gynes' future reproduction potential, and supports males' spermatogenesis, which occurs only during larval development (Machida 1934).

Indirect genetic effects
Indirect genetic effects (IGEs) are environmental influences on the phenotype of one individual due to the expression of genes in a different, conspecific individual (Wolf et al. 1998). IGEs are ubiquitous in social insects (Linksvayer 2015) and foundational to fitness values of every individual. Each dominance interaction, nourishment transfer, and bout of antennal drumming has an IGE (Fig. 3) in which one wasp's reproductive potential is enhanced while diminishing another's. IGEs change dynamically and continuously across the colony cycle. Related workers may gain an indirect fitness return (Fig. 3A), whereas unrelated workers cannot (Fig. 3B). A queens' direct fitness is undiminished by unrelated allomaternal workers Fig. 1 A The annual cycle as typically presented from a colony-level perspective. Founding Phase: the annual cycle begins (arrow) when a gyne of the previous generation emerges from quiescence, feeds on nectar and caterpillar hemolymph, develops her ovaries, founds a nest, and initiates oviposition and maternal care of her larvae. Worker Phase: the first female offspring to emerge in early summer are nonreproducing allomaternal workers that forage, feed nestmate larvae, and construct the expanding nest. Reproductive Phase: female offspring emerging in mid-summer are non-working future foundresses called gynes. Males emerge synchronously with gynes. Decline Phase: the queen and workers die, gynes and males depart the empty nest to mate and feed at fall flowers. Quiescence Phase: males die, and gynes enter a torpor-like quiescence in sheltered concealment. Following quiescence, the next cycle begins. B The colony cycle from an individual physiological-level perspective. The synovigenic foundress emerges from quiescence with pre-reproductive physiology. She feeds on nectar and caterpillar hemolymph as oocytes develop in her ovaries, initiates a nest, begins oviposition, and performs the complete module of maternal care behaviors until emergence of her first daughters. She then becomes queen, ceases to forage, and increases her oviposition rate in response to nourishment brought by allomaternal foragers. First-emerged daughters have pre-reproductive physiology and synovigenic ovaries, but energetic costs and physiological consequences of allomaternal care lead to facultative sterility. Female larvae that receive elevated feeding levels from multiple allomaternal caregivers emerge as gynes in reproductive diapause with substantial stores of nourishment, which enables them to pass an unfavorable season (cold, hot and dry) in quiescence in a sheltered place. Stored nourishment is diminished or depleted preceding emergence to become foundresses of the next annual generation ◂ rearing her reproductive offspring, showing that "socially acting genes" and "sib-social care" occur independently of relatedness and are merely context-dependent expressions of maternal care genes and behavior.

Diapause ground plan
The ability of haplodiploid females to determine sex of their offspring is reflected in alternating sex ratios between the two generations of partially bivoltine solitary wasps and bees (Seger 1983;Brockmann and Grafen 1992). The first generation is male-biased and has passed developmental diapause as final instar larvae (prepupae) within closed nest cells until the start of the next favorable season, when they emerge to mate and lay eggs for the second generation. The second generation has a balanced sex ratio when it emerges, mates, and lays eggs for larvae that enter developmental diapause at the end of larval growth.
The Diapause Ground Plan Hypothesis (DGPH) (Hunt 2006) proposes that dichotomization of worker and gyne castes in eusocial Hymenoptera reflects the same ancestral physiological mechanisms and pattern of alternating nondiapause and diapause generations seen in bivoltine solitary wasps, but the physiological and developmental pattern occurs among adults within a single life cycle (Hunt and Amdam 2005). Sex ratios in the two generations also have changed. In Polistes, diapause is female-only and is passed in a state of behavioral quiescence. A post-diapause foundress/queen rears a female-biased brood of first offspring that pass development and emerge from pupation with prereproductive physiology and become allomaternal caregivers that then rear a queen's balanced sex ratio second brood of males and gynes, with gynes in reproductive diapause   (Fig. 1). Gynes in reproductive diapause pass an inclement season in behavioral quiescence prior to becoming foundresses in the following favorable season. Among paper wasps and in addition to Polistes, a female reproductive diapause phenotype occurs in some species of Belonogaster (Keeping 2002), Parapolybia (Saito-Morooka 2014), Mischocyttarus (Hunt et al. 1999), and Ropalidia (Saito and Kojima 2005). Several tropical species in locales with wet/dry seasonality have a diapause ethotype as in temperate zone species as part of the colony cycle (Hunt et al. 1999;González et al. 2005). Gobbi et al. (2006) describe "winter" aggregations and the same colony cycle in a subtropical Polistes. Colonies at all stages of development occur throughout the year in several tropical species of Mischocyttarus (Jeanne 1972;Litte 1981;O'Donnell and Joyce 2001) and Polistes canadensis (Southon et al. 2019), suggesting that diapause/non-diapause physiological dichotomy may not occur in lowland tropical locales, however diapause phenotypes do occur in Polistes canadensis near the end of the wet season (Giray et al. 2005).

Sex ratio
All Hymenoptera have haplodiploid sex determination in which diploids are female and haploids are male. At the time of oviposition, plasticity of fertilization as ova pass the duct to the spermatheca, where sperm are held, enables an ovipositing female to control the sex ratio of her offspring. Sex ratios vary widely across the order (parasitoids: Hardy 1994; solitary wasps: (Danks 1983; eusocial Hymenoptera: Gardner and Ross 2013). Paper wasps, indeed all eusocial wasps, are protogynous and produce female offspring prior to males. Females with reproductive or pre-reproductive physiology are the only paper wasps that engage in maternal or allomaternal larval care. A queen that includes males in her early brood would thereby reduce her prospects of producing a reproductive brood when compared to a queen with an initial all-female allomaternal brood that rears her Caregiver Fig. 2 The solitary foundress feeds prey tissue and hemolymph to her developing allomaternal larvae, which in turn produce a nourishing saliva on which she feeds. When allomaternal caregivers emerge from pupation, the foundress transitions (dotted arrow) into the queen, which continues to nourish herself on saliva of larvae plus nectar and prey hemolymph taken from allomaternal foragers. When gynes emerge from pupation, they nourish themselves from the same sources as the queen. Solid arrows denote major pathways; dashed arrows indicate pathways of uncertain but likely minor significance. These nourishment transfers are foundational for the social dynamics shown in Fig. 3 larvae that become her direct fitness comprised of gynes and males produced later in the colony cycle.

Demography
At the population level, high colony loss is mitigated by a minuscule percentage of nests that reach a large size and produce large numbers of gynes (Wenzel 1989;Hunt 2007). In consequence of this, the main driver of paper wasp population dynamics is the role played by allomaternal workers to yield the highest possible number of gyne offspring at the end of the colony cycle, giving the queen the best possibility of a daughter surviving the gauntlet of ecological mortality factors to become one of the minuscule number of successful queens in the following generation. A foundress without allomaternal caregivers cannot rear gynes (West Eberhard 1969), therefore eusociality in paper wasps can be considered to be obligatory (O'Donnell 1998b;Seppä et al. 2012;Hunt and Toth 2017). Three minor exceptions have been noted. Fucini et al. (2009) documented inter-population life cycle variation in populations of Polistes biglumis in which two populations at high elevations with cold temperature regimes and heavy selection pressure from an interspecific cleptoparasite had only a single offspring generation, all with gyne-like characteristics, thus indicating the virtual absence of workers. In a population of Polistes aurifer in which 86% of nests have solitary foundresses, Liebert et al. (2005) found that due to foundress turnover and nest foundation in later parts of the season, many nests produced only one set of offspring, resulting in a loss of the eusocial nature in those nests. To be clear, the P. biglumis and P. aurifer examples represent only portions of populations of the same species that otherwise express simple eusociality. Ropalidia rufoplagiata has a social structure apparently unique among paper wasps (Sinha et al. 1993). Thirty-three of 46 identified females on a single colony in subtropical India were observed to oviposit from 1 to 17 times over a 45-day observation period. No predominant egg layer was  Fig. 2, showing indirect fitness effects (IGEs) among colony members of a complete colony cycle. Letters z are phenotypes that reflect the fundamental theorem of quantitative genetics, z = a + e, where a is the additive genetic variance, and e is an environmental effect. Subscripts indicate life cycle phenotypes: z if and z i for the foundress/queen of generation i, z ja for the queen's allomaternal daughters, z jg for her gyne daughters, z jf and z j for foundresses and queens of generation j at the start of the next colony cycle, and z w for unrelated allomaternal workers. The environment, e, of the fundamental theorem can be partitioned into a general environmental effect, e, and the environmental effects contributed by interacting phenotypes. Thus, the environment for the queen z i becomes e + e ja + e jg . Assuming that e is equal for all members of a colony, the environment for phenotype z i reduces to e ja + e jg . Environments of other individuals or groups are similarly defined. Additive genetic variances are not shown to more clearly present IGEs. A IGEs are denoted by Ψ with subscripts of the format: affected individual(s), affecting individual(s), proportion of the affected individual(s)' phenotype as a consequence of the affecting individual(s)' phenotype. The interaction effect coefficient Ψ jaiZi therefore describes the extent to which phenotype z ja is changed by interactions with phenotype z i (notation after Moore et al. 1997). Blue arrows denote a generalized maternal effect. Mauve arrows and subscripts denote nourishment transfers, which have positive effects on recipients. Negative consequences of those nourishment transfers, including foraging energy costs, are denoted by violet Ψ subscripts. Dark orange arrows and subscripts represent dominance interactions. Queen dominance has negative effects on allomaternal caregivers. The light orange Ψ subscript denotes positive feedback to the queen of her dominance. Dominance interactions among allomaternal caregivers give rise to a dominance hierarchy. Black solid arrows denote the queen's fitness. Dashed black arrows represent phenotype transitions. The gray arrow denotes allomaternal caregivers' indirect fitnesses. B Phenotype z p represents a facultative or obligate social parasite that usurps the nest of an unrelated queen (Table 1, #s 5 and 12). Phenotype z i represents a successful reproductive in any of 12 contexts in which her future reproductive offspring derive direct fitness benefits from unrelated allomaternal caregivers (Table 1). Arrows and Ψ elements are color coded as in A. Allomaternal caregivers' fitnesses are zero. The direct fitness of the queen is undiminished from that in A detected, suggesting the absence of a permanent reproductive caste in this species.

Diversity of unrelated workers
Natural history studies of Polistes have documented at least twelve contexts in which allomaternal behavior is directed to larvae of an unrelated queen or other egg-laying member of the colony (Table 1). Although colony members were related at the ancestral dawn of the allomaternal non-reproductive worker phenotype, providing a quintessential relatedness structure for kin selection, allomaternal care by non-relatives in extant species suggests that workers were not the target for selection. Allomaternal caregiving daughters at the threshold of paper wasp eusociality could have been, in concept, substituted by unrelated allomaternal caregivers with no fitness consequence for the queen (Fig. 3). The paper wasp worker phenotype could have originated exclusively through allomaternal behavior of first emerged daughters in response to sign stimuli from nestmate larvae. This would have been accentuated by maternal manipulation via dominance by the foundress leading to enhanced nutritional exploitation of her reproductively competent daughters. At the origin of the allomaternal worker phenotype, fitness benefits accrued only to the queen.

Experimental validation
Experiments have validated aspects of Polistes development and life history described above. In natural environments, supplemental nourishment of foundress nests yielded firstemerging offspring with gyne-like physiological characteristics (Rossi and Hunt 1988). Supplementation for the full colony cycle yielded more female offspring during the colony phase of gyne production (Seal and Hunt 2004); lower oophagy, more rapid nest growth, more offspring, longer worker lifespan, and lower rates of larval cannibalism near the end of the colony cycle (Mead and Pratte 2002); and caste-related differences in nutrient levels (Judd et al. 2009). Simulated antennal drumming during the gyne production phase of the colony cycle yielded offspring with worker-like characteristics (Suryanarayanan et al. 2011b). Restricted foraging combined with simulated antennal drumming decreased colony growth due to larval disappearance (Jandt et al. 2017). Diminishment of larval saliva yielded extremely low colony survivorship, colony growth, and number of offspring produced (Hunt and Dove 2002). Table 1 Twelve contexts in which Polistes workers care for larvae to which they are unrelated 1: A nest that has lost its foundress (an orphan nest) can be opportunistically adopted by a different wasp of the same species, and workers that develop from eggs laid by the foundress rear reproductive offspring of the adopting queen (Seppä et al. 2012) 2: If a nest loses its foundress after some larvae pupate but prior to worker emergence, orphan offspring that emerge can move to another colony ("drift") and rear offspring of that colony's queen (Hagiwara and Kojima 2002) 3: Recently emerged potential workers can drift from their natal nest to a nearby nest, even though their natal colony is functioning normally (Kasuya 1981) 4: Intra-specific nest usurpation during the foundress phase is widespread, apparently as a consequence of predation on nests by birds and mammals that leaves many foundresses without the nest they founded (Klahn 1988). The invading queen eats the original queen's eggs and small larvae, which would have become her reproductive offspring, and replaces them with her own (Klahn 1988;Makino and Sayama 1991).
Offspring of the usurped wasp would then rear reproductive offspring of the usurper 5. A wasp that is capable of independent nesting can "sit and wait" without founding a nest and instead displace a conspecific foundress from her nest once it is established. The usurped wasp's offspring become workers then rear reproductive offspring of the usurping wasp (facultative cleptoparasitism) (Starks 1998) 6: A subdominant conspecific co-foundress can allomaternally rear larvae of an unrelated dominant co-foundress (the colony queen) (Queller et al. 2000) 7: A subdominant conspecific co-foundress unrelated to the queen can become the new queen following disappearance of the initial queen, and allomaternal offspring of the initial queen rear offspring of the second queen (Leadbeater et al. 2011) 8: A subdominant conspecific co-foundress can successfully oviposit in an empty nest cell with an unrelated colony queen (Leadbeater et al. 2011) 9: "Hypergynous" colonies occur when a large number of co-foundresses utilize an empty nest from the previous season, with multiple queens and their offspring functioning as a single colony (Liebert et al. 2008) 10: Two species can co-found a nest, and one co-foundress works to rear the offspring of the other species (O'Donnell and Jeanne 1991) 11: A wasp of one species can occupy the orphan nest of another species (Hunt 2009) 12: Foundresses of three Polistes species (Choudhary et al. 1994) are obligate cleptoparasites of Polistes dominula that "sit and wait" until the P. dominula queen species has reared a small worker force. A cleptoparasite queen then invades that nest and dominates or kills the queen and usurps her nest and allomaternal offspring (Cervo and Turillazzi 1996). The obligate cleptoparasites have thicker and stronger mandibles, thicker integuments, strongly developed forelegs, and larger heads (Cervo 1994). Cleptoparasites have gyne and male offspring but are dependent on the host species' workers to rear them. The cleptoparasites have no worker offspring Individuals emerging onto nests without larvae, therefore without stimuli that elicit allomaternal care, showed signs of active reproduction and nest construction (Judd 2018). In the laboratory, augmented nourishment increased gyne-like characteristics (Karsai and Hunt 2002;Judd et al. 2015); restricted nourishment increased larval cannibalism by foundresses (Kudô and Shirai 2012); and exposure to cold revealed physiological and survival differences between workers and gynes (Strassmann et al. 1984;Karsai and Hunt 2002). Facial patterns of Polistes dominulus from colonies reared with supplemental nourishment signaled higher aspects of quality related to dominance (Tibbetts and Curtis 2007). Non-supplemented colonies of both P. dominulus and Polistes metricus in that same experiment produced smaller offspring than in supplemented colonies (Tibbetts and Curtis 2007). Restricted nourishment in a different experiment with P. metricus yielded smaller offspring (Karsai and Hunt 2002). P. metricus on a restricted nourishment treatment fed more on sugar, had lower aggression, and had a higher net loss of larvae than those on supplemented diets, which fed more on caterpillars, had higher aggression, and lower net loss of larvae (Walton and Toth 2021). Experimental transfer of paper wasp foundresses to pre-emergence nests of unrelated foundresses replicated characteristics of intra-specific facultative cleptoparasitism (Cervo and Turillazzi 1989). In combined field and laboratory studies of Mischocyttarus pallidipectus, workers with higher dominance indices spent more time on the nest, had a diet higher in animal-based sources, and had greater ovary development than workers with lower dominance indices (Fiocca et al. 2020).
Theoretical support Kay et al. (2020) scrutinized formal evolutionary models of 119 papers that investigate dynamics of either genetic variants (alleles) or cultural variants (memes), with most concentrating on only two competing variants (one "altruistic" and one "selfish"). Criteria for the evolution of altruism were identified in 89 of the models and cooperation in 28. Kay et al. (2020) therefore infer that relatedness plays an implicit role in the models, even in those for which an explicit role for relatedness isn't stated or even inferred by authors of the models. Bourke (2011, p. 38) asserts, "altruism cannot evolve when relatedness is zero, i.e., when the recipient [of altruistic aid] is unrelated." However, even though allomaternal worker behavior typically arises in the context of social relatives, that does not mean that an underlying relatedness model is correct. Using a quantitative genetics approach, Bijma and Wade (2008) developed models in which the trait value of an individual is modeled as the sum of a direct genetic effect to that individual and phenotype consequences caused by IGEs of other individuals interacting with the focal individual. Among other conclusions, they show that IGEs create the opportunity for the evolution [origin] of an altruism phenotype without the need for relatedness or multilevel selection.
Also using a quantitative genetics approach, Moore et al. (1997) present a family of models in which IGEs affect phenotypic evolution of traits expressed during interactions among unrelated conspecific social partners. In a model in which two interacting individuals express different traits affected by expression of the other trait, exemplified by dominance and subordinance, higher levels of dominant behavior in one actor increases the expression of subordinate behavior in the other actor and vice versa. Dominance interactions among related as well as unrelated Polistes individuals would be an empirical example that conforms to their model. González-Forero and Gavrilets (2013) use techniques derived from the Price (1970) equation to investigate the evolution of manipulated behavior. Their hypothetical example addresses "the evolution of eusociality" via maternal manipulation as it would be present in a colony of paper wasps with two broods of offspring. Individuals exist in one of three states: manipulator (queen), manipulated (first brood offspring), and target of manipulation (second brood offspring). In the first brood, resistance is expressed as leaving the colony, thereby escaping manipulation. Acquiescing offspring remain and may expresses allomaternal care toward the second brood, thereby increasing fitness of the queen and her future reproductive offspring. They find that manipulated behavior can evolve [originate] and be maintained with zero relatedness between actors and recipients if resistance is costlier than acquiescence.

Discussion
The novel phenotype of paper wasp allomaternal non-reproductive workers is comprised of reproductively competent females with undeveloped synovigenic ovaries, initially constrained to remain at their natal nest by an inability to fly and induced to return to it by the attractiveness of larval saliva as nourishment source for potential ovary development. At the nest, they respond to sign stimuli from nestmate larvae that release the module of maternal behaviors directed allomaternally toward larvae that are not their own. The energetic costs of allomaternal care constrain their ovary development, rendering them functionally sterile.
This phenotype would have first appeared within matrifilial colonies, thus providing a framework for response to selection (Southon et al. 2019). However, the many ways in which allomaternal care is mis-directed to non-relatives with no alterations in performance indicates that relatedness between allomaternal workers and a queen's fitness-yielding reproductive offspring was not the target of selection. Evolutionary processes at the ancestral dawn of the allomaternal worker phenotype, such as selection favoring protogeny and dominance behaviors, were driven by direct selection on the queen. Allomaternal worker daughters could, in concept, have been replaced by allomaternal workers unrelated to the queen with no diminishment of her fitness.
Allomaternal care of nestmate larvae in paper wasps has a social structure analogous to alloparental behavior of songbirds feeding a cuckoo nestling. When the songbirds' offspring in their natal nest present sign stimuli of open mouths and food begging calls, the adults respond with parental care. If the nestling is that of a cuckoo, sign stimuli of an open mouth and food begging call stimulate the adults to respond with alloparental care. Just as in paper wasps, parental and alloparental care by songbirds are identical responses to sign stimuli that are independent of relatedness between caregiver and recipient. The fitness beneficiary in paper wasps is the individual that laid the egg, whether or not the allomaternal caregivers are related to her and her future reproductive daughters and sons. In cuckooed songbirds, the fitness beneficiary is the cuckoo that laid the egg that became the begging chick that received alloparental care. The analogy is such that interspecific Polistes cleptoparasites are sometimes called cuckoo paper wasps (Cervo and Turillazzi 1996).
Gyne paper wasps enter reproductive diapause at the same point in the life cycle every generation, a feature that indicates it to be an obligate feature of the life cycle (Hahn and Denlinger 2007). Indeed, natural history studies suggest that every lineage of Hymenoptera that contains eusocial species also contains solitary species that have developmental diapause (Brockmann 1984, Santos et al. 2019. Developmental physiology of bivoltine solitary vespid wasps, i.e., reproductive physiology of first-generation adults and developmental diapause of the second generation, has been hypothesized to be the same as that in paper wasps, in which first brood female workers have pre-reproductive physiology and gynes have reproductive diapause physiology (Hunt and Amdam 2005). Comparative physiological studies of bivoltine solitary wasps vs. wasps with simple sociality could be informative regarding the life cycle transition from prepupal diapause in solitary wasps to reproductive diapause in gyne paper wasps. Treanore et al. (2020) document a gradual change in physiology of bumble bee queens' offspring from reproductively capable early in the colony cycle to diapauselike in later phases of the cycle. They opine that existing data and examples are sufficient to recommend that the diapause ground plan hypothesis warrants further direct investigation.
Boomsma and Gawne (2018) present a comprehensive review and analysis of an extensive literature addressing the diverse definitions and concepts of species with complex eusociality. They posit that hypersociality sensu Batra (1995) and obligate eusociality sensu Crespi and Yanega (1995) and Boomsma (2009) are functionally identical to superorganismality sensu Wheeler (1928). Superorganismal colonies are those that have crossed an irreversible major transition and are characterized by permanent morphological castes, distinct lifetime matedness difference between queens and workers, and queens domesticated by workers. Using those criteria, vespine wasps (yellowjackets and hornets), despite having solitary foundresses, are superorganismal. Boomsma and Gawne (2018) place paper wasps in the middle of their spectrum from cooperative breeding to eusocial. Characteristics that determine that placement are alpha queens that maintain dominance hierarchies and capability of all females to mate, thus they have not crossed a major transition to become superorganismal. There are apparent exceptions to paper wasps' placement in the Boomsma and Gawne sociality spectrum. Although most paper wasps show a gradation of sizes without morphological castes (Haggard and Gamboa 1980), a handful of species have discrete worker/gyne caste dimorphism: Ropalidia ignobilis (Wenzel 1992), Belonogaster griseus (Pardi and Marino Piccoli 1981), Belonogaster peteolata (Keeping 2002), and Polistes olivaceous (Alam 1958;Kundu 1967). Crossing the divide from monomorphism to dimorphism can only have evolved via selection acting on colonies containing allomaternal workers related to the queen's reproductive brood that they rear. These species may therefore have crossed the major transition threshold and become superorganismal.
Paper wasps with simple eusociality and vespines (hornets and yellow jackets) with complex eusociality share a common ancestor and are unrelated to social wasps in Stenogastrinae Moritz 1998, 2000;Hines et al. 2007;Bank et al. 2017;Peters et al. 2017;Piekarski et al. 2018). Reproductive queen and worker differentiation appeared in the common ancestor (Piekarski et al. 2018), but the common ancestor would not have had morphological castes (contra speculations by Piekarski et al. 2018). The queen of a paper wasp colony is entirely dependent on allomaternal workers to rear her reproductive offspring, therefore paper wasps with simple eusociality also have obligate sociality (O'Donnell 1998b;Seppä et al. 2012;Hunt and Toth 2017). This is the case whether or not allomaternal workers are related to the queen or other colony members. Paper wasps' obligate eusociality occurs in the absence of superorganismality. The evolutionary force that results in paper wasp obligate eusociality is direct selection acting on the foundress/queen. Solitary foundresses of hornets and yellow jackets have complex eusociality characterized by queen-worker dimorphism, which can only have evolved in colonies of relatives. As in paper wasps, a vespine queen's reproductive success is entirely dependent on the presence of workers, therefore vespines' eusociality obligate. However, occurrence of several species of obligate social parasites (Carpenter and Perera 2006) indicates that, as in paper wasps, relatedness between workers and the queen's reproductive offspring is not required for a queen's reproductive success.
New families of models could expand our understanding of simple eusociality dynamics regardless of relatedness between allomaternal and beneficiary colony members. The ability to specify directionality of IGEs (Fig. 3) is a novel addition to the Moore et al. (1997) IGE models. Fiocca et al. (2020 present a social network diagram (their Fig. 2) with quantified dominance pathways in a large colony of Mischocyttarus pallidipectus paper wasps. The diagram shows that it should be possible to use a combination of IGE and individual-based modeling to quantify variations in eusocial outcomes based on parameterized IGEs, at least in colonies with fewer individuals. A general individual-based model (Judson 1994) with variables drawn from the model in Fig. 3 could be combined with a model for the timing of reproduction in annual plants with two-stage life cycles (Schaal and Leverich 1981) to address the paper wasp population survivorship pattern conjectured by Hunt (2007, p. 128). In so doing it could test the proposition that allomaternal workers are essential to a queen's fitness and that simple eusociality is indeed obligate. Linksvayer and Wade (2005) hypothesized the presence of heterochrony genes that can modify the timing of expression of maternal care as part of a scenario for the evolutionary origin and elaboration of eusociality, i.e., the evolution of allomaternal workers. As has been outlined here, however, it is not necessary to invoke a special set of genes-or any genes-to explain the origin of the allomaternal non-reproductive worker phenotype. Marshall (2016, p. 103) noted that "[c]ritiques of inclusive fitness have become increasingly mathematical in recent years, as have their defences." This contest of theoretical ideas proceeded without heed to the insight of Thomas Kuhn, who pointed out that "[t]he competition between paradigms is not the sort of battle that can be resolved by proofs" (Kuhn 1962, p. 391). In a paper entitled, "What is inclusive fitness theory, and what is it for?", Marshall (2016, p. 103) made reference to Kuhn and opined, "if the ongoing battle is to be 'put to bed' then the resolution will be conceptual, not mathematical." Pursuant to Hamilton' influential publications (Hamilton 1964a, b) in which he introduced the concept of inclusive fitness, accepted wisdom is that eusocial insect workers altruistically sacrifice direct reproduction as a means to maximize their inclusive fitness. "In social groups, only a few individuals often monopolize actual reproduction. The others more or less voluntarily assume the role of helpers whose fitness is dependent on helping relatives, not on reproducing themselves" (Seppä, Queller and Strassmann 2012). When Wilson (1971) broadly disseminated Hamilton's ¾ relatedness hypothesis for "the evolution of eusociality," he commingled the origin of eusociality with its subsequent elaboration. The distinction is critical. Hamilton (1972) recognized that selection processes in eusocial origins may differ from those affecting its maintenance or elaboration. His insight on that point passed largely unnoticed, thereby facilitating decades of publication in which a distinction between the two has been blurred or, mostly, altogether missing. Pernu and Helanterä 2019, p.1) point out that, "[a] lthough reviewing the empirical evidence concerning the evolution of eusociality reveals that relatedness does not play a role in the initial appearance of helper phenotypes, this follows simply from the fact that natural selection-of which relatedness is a necessary component-does not play a causal role in the origin of any traits." The paper wasp allomaternal non-reproductive worker phenotype did not evolve from a foundation of relatedness. Instead, it originated from a confluence of multiple proximate factors in paper wasp biology. However, the evolutionary elaboration of eusociality requires relatedness. In the history of eusociality, an allomaternal non-reproductive worker phenotype had to originate in a context of simple eusociality before the evolution of complex eusociality could begin.

Conclusion
Paper wasps' allomaternal non-reproductive worker phenotype originated as a context-dependent expression of maternal behavior performed allomaternally and serving a new function within the ecology of paper wasps. The nexus of nest architecture, larval provisioning behavior, larval development, and reproductive and diapause physiology, in combination with IGEs arising from nourishment and social dynamics creates an "exaptation" (Gould and Vrba 1982) that gave rise to the novel phenotype. Relatedness between allomaternal non-reproductive workers and the larvae they care for played no role. A large literature, both natural history and experimental, has enabled insights and synthesis leading to this conclusion. Using paper wasps as a focal example demonstrates that empirical studies of factors that affect the origin of an allomaternal non-reproductive worker phenotype in simple eusociality can offer a new perspective on the foundation of complex eusociality, one of the most remarkable phenomena in evolution.

3
and three anonymous reviewers. I dedicate this paper to Vija Sherman and thank her for her steadfast support and invaluable editorial acumen.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/.