Introduction

Adaptation to different environmental conditions via divergent natural selection can generate phenotypic and genetic differences between populations, which may in the absence of gene flow over time lead to the formation of new species (Nosil 2012). During speciation, divergence in phenotypic traits, genetic constraints, and reproductive isolation varies quantitatively; speciation thus represents a continuum of divergence, starting from continuous variation to distinct species pairs at the final stage (Nosil 2012). Dependent on the extent and stability of reproductive isolation, incipient phenotypic differentiation can lead to the formation of distinct phylogenetic lineages, but this is not necessarily the case (Lowry 2012). Intermediate stages in ecological speciation involve ecotypes (Lowry 2012), i.e. groups of populations within one species differentiated due to adaptation to local environmental conditions, which are frequently still interfertile (Hufford and Mazer 2003) and result from a combination of heritable and non-heritable traits (Pfennig et al. 2010; Bonduriansky et al. 2012). Differences in phenotypic traits associated with increased fitness in a specific habitat can result from environmental conditions experienced during ontogeny (phenotypic plasticity) but can also be mediated by heritable (epi-)genetic differences (local adaptation via phenotypic differentiation, Flatscher et al. 2012). In the latter case, non-adapted immigrants and hybrids exhibiting intermediate phenotypic traits may show reduced fitness (Rice and Hostert 1993; Rundle and Whitlock 2001).

Reproductive isolation is a critical step in speciation as it prevents hybridisation between diverging populations (Nosil 2012) and thus plays an important role in maintaining discrete phenotypic groups (Rieseberg et al. 2006). Nature and extent of reproductive barriers acting at the onset of speciation vary strongly among plant species (Lowry et al. 2008a; Baack et al. 2015), which prevents generalisation of the underlying fundamental processes. Isolation can be caused by prezygotically and/or postzygotically acting barriers (Lowry et al. 2008a; Baack et al. 2015) and may either be intrinsic or extrinsic (Baack et al. 2015). Prezygotically, intrinsic barriers can for example prevent interspecific pollination and/or pollen tube growth by morphological or physiological incompatibilities (Baack et al. 2015), whereas extrinsic isolating barriers may be imposed by spatial isolation of potential mating partners due to ecological preferences (Wang et al. 1997; Coyne and Orr 2004). Postzygotically, development, viability and/or fertility of hybrids may be impeded by intrinsic developmental and physiological problems (i.e. hybrid breakdown), whereas extrinsic barriers involve fitness disadvantages of hybrids in the parental environments (Wang et al. 1997; Coyne and Orr 2004).

An excellent example for exploring the extent of reproductive isolation during early stages of the speciation continuum are species of Heliosperma (Caryophyllaceae) from the Alps. Heliosperma veselskyi Janka is restricted to a few scattered populations growing below rock overhangs in the montane belt (500–1300 m a.s.l.) of the south-eastern Alps. In contrast, its widespread close relative, H. pusillum (Waldst. and Kit.) Rchb. occurs on humid rock faces and screes in the montane and alpine zone (1700–2300 m a.s.l.) throughout the southern and central European mountain ranges. Both taxa have been described at the species level and are treated as independent species in national floras (Poldini 2002; Fischer 2008). H. veselskyi is characterised by a dense indumentum of long multicellular glandular hairs, whereas H. pusillum is glabrous or has only sparsely hairy leaves, often bearing unicellular glands (Neumayer 1923; Frajman and Oxelman 2007). However, DNA sequence data (Frajman and Oxelman 2007; Frajman et al. 2009) as well as Restriction site Associated DNA (RAD) markers (Trucchi et al. bioRxiv preprint, available at doi:10.1101/044354) suggest that the two species are phylogenetically not distinct. As the disjunct populations of H. veselskyi are inextricably nested within the widespread H. pusillum, H. veselskyi rather represents a habitat specific ecotype, which evolved at multiple localities in parallel in response to similar environmental conditions. For the sake of simplicity, we here treat H. pusillum and H. veselskyi at the species level in spite of the highly debatable taxonomic value of the latter.

As a phylogeographic study employing RAD markers (Trucchi et al. bioRxiv preprint, available at doi:10.1101/044354) found almost no evidence for gene flow between geographically close populations of H. pusillum and H. veselskyi, we here explore the mechanisms causing the observed reproductive isolation in two population pairs. (1) First, we tested if the lack of gene flow is a result of intrinsic reproductive isolation. We evaluated existence and extent of intrinsic reproductive barriers by controlled pollinations in a common garden and consecutive measurement of offspring fitness. (2) Second, by applying morphometric analyses to offspring of intra- and interspecific crosses we investigated whether phenotypic differentiation between H. pusillum and H. veselskyi is heritable—indicating the onset of speciation—and whether hybrids exhibit morphologically intermediate phenotypes. Breakdown of phenotypic differentiation in intraspecific crosses grown in a common garden would suggest that the morphological differentiation of phenotypes is caused by phenotypic plasticity. Further, intermediate morphology of hybrids may confer maladaptation to parental growing sites and thus allows insights into the fitness reduction at natural growing sites.

Materials and methods

Controlled crosses

Controlled cross pollinations were conducted in 2013 in a common garden (Innsbruck, Austria, 610 m). Plants were grown from seeds collected in the field in autumn 2012 in two regions containing geographically close populations of H. pusillum (P) and H. veselskyi (V): Austria, Lienzer Dolomiten: Mt. Rudnigkofel, 46.762°N, 12.877°E, 2056 m (P) and Anetwände, 46.774°N, 12.901°E, 789 m (V); Italy, Dolomiti Friulane, Val Cimoliana: 46.391°N, 12.480°E, 1701 m (P) and 46.380°N, 12.489°E 1182 m (V). Intraspecific (P–P, V–V) as well as interspecific crosses in both pollen donor/pollen recipient combinations (P–V, V–P) were conducted, resulting in four pollination treatments, which were replicated for both geographic regions, resulting in eight crossing groups. For each crossing treatment 34–46 potted individuals were used as pollen recipients (Table 1). Several flowers in bud stage were emasculated on each of these plants to avoid self-pollination. Pots were then bagged with a fine mesh fabric to avoid undesired pollen transfer by insects. After bud opening, stigmas of emasculated flowers were brushed with flowers of at least five pollen donors randomly selected out of a stock of 160 additionally cultivated and bagged individuals. To avoid seed loss and ensure complete maturation of seeds, capsules were covered with a fully gas permeable tape (3M Transpore, North Coast Medical, Gilroy, US). Harvested seeds were stored in paper bags at 4 °C. Seeds were visually classified as fully developed (firm and plump testa) and empty (empty testa). Seed set, defined as the proportion of fully developed seeds, and mean seed weight were estimated for each capsule.

Table 1 Number of individuals at various developmental stages in cross-pollination and consecutive fitness experiments of H. pusillum (P) und H. veselskyi (V)
Full size table

Germination experiments

In February 2014, 120 fully developed seeds per pollination treatment (i.e. 60 seeds per pollination group) were sown on the surface of pots filled with a mixture resembling natural soils, composed of compost, ground earth, lava, turf, quartz sand, rock flour and pumice gravel. Pots were exposed to approximately natural conditions in two climatic chambers (Percival PGC-6HO): day/night 16/8 h, temperature 15/5 °C, approximately 800 µmol photons m−2 s−1 light during day time. Pots were distributed randomly and repositioned two times a week within and between climate chambers to avoid edge effects. Soil was kept humid throughout the experiment and pots were monitored two times a week to record emergence of the radicula (in the following referred to as germination), the cotyledons, the first leaf pair as well as the second, third, and fourth leaf pair of seedlings in the F1 generation. Two months after germination, size measures including plant height, diameter of the ground rosette, number of leaves, length of the longest leaf, and number of flower buds were recorded. Except for ten randomly chosen plants of each crossing group (see below), seedlings were cut at the base of the stem and dried at 60 °C for 72 h to determine their dry weight. The retained plants were cultivated in a common garden allowing free pollination to test if viable seeds were produced. From these, 100 fully developed seeds per crossing group—except for V–V from region Dolomiti Friulane, where only 89 fully developed seeds were available—were harvested and used in a further germination experiment, which was conducted in spring 2015. Seeds were germinated in the above mentioned soil mixture in a common garden and the number of seedlings in the F2 generation was counted after 8 weeks.

Success of inter- (P–V, V–P) versus intraspecific (P–P, V–V) cross-pollinations was quantified based on the following six fitness parameters: seed set expressed as the proportion of fully developed seeds, mean seed weight of individuals in the F0 generation, germination rate, time from seeding to the emergence of cotyledons in days, seedling weight of individuals in the F1 generation, and germination rate in the F2 generation.

The effect of pollination treatments on each of these fitness parameters was analysed separately with mixed-effects models. A linear mixed-effects model (LMM) was employed for mean seed weight, whose residuals were normally distributed and with generalised linear mixed-effects models (GLMM) for the other fitness parameters. To model the time to emergence of cotyledons and the seedling weight, whose residuals followed a Poisson distribution and a gamma distribution, respectively, we applied the canonical logit link and the inverse-link function, respectively. For the germination rates in the F1 and F2 generations as well as the seed set, whose residuals were binomially distributed, we applied a logit-link. Parameters were estimated by restricted maximum likelihood in case of the LMM and based on the Laplacian approximation in case of the logistic and the Poisson GLMM. For all models pollination treatment was used as the only fixed-effect predictor. As we used seeds originating from two distinct regions and cross-pollinated several flowers of each pollen recipient plant, region of origin and the pollen recipient plant nested within region were included as grouping variables, and a random intercept for each group was allowed. For the germination rate in the F2 generation, solely region of origin was included as random factor, as seeds were pooled for all F1 individuals of a population subjected to a particular pollination treatment. Seed set was modelled using the total number of (fully developed and empty) seeds per capsule as weighting factor to account for differences in capsule size.

Unequal suitability of artificial conditions in ex situ experiments might bias results towards one crossing partner. Thus, we compared the performance of interspecific cross-pollination treatments (i.e. P–V or V–P) to those of the pooled intra-specific cross-pollinations (i.e. P–P and V–V) reflecting a test of the hypothesis that in case of no selection against hybrid offspring its fitness should be intermediate to those of the parental species. These tests were done by splitting the data into two subsets (the first including P–P, V–V and P–V, the second comprising P–P, V–V and V–P) and applying Helmert contrasts separately to each subset of each fitness parameter. We applied Bonferroni-correction to account for multiple comparisons.

Because seedling weight could not be measured for plants further cultivated, we used predictions of a LMM parameterised using plants harvested and weighed at the end of the experiment. We fitted models using the species of the pollen recipient and the pollen donor plant as well as (combinations of) size measures as fixed effects. Model selection based on pseudo-R2 (McFadden 1974) identified the model with the first and second orthogonal polynomial of longest leaf as the most suitable to predict seedling weight. We used the same structure of random-effects as described above. Measured seedling weight was transformed using a BoxCox-Transformation with lambda 0.02 to approximate normal distribution.

Morphometric analyses

Based on determination keys (e.g., Fischer 2008), descriptions in the literature (e.g., Neumayer 1923) and personal observations, 27 morphological characters were chosen and measured on herbarium vouchers of shoots of 66 randomly chosen plants of the F1 generation, including plants of all crossing groups. Shoot characters, i.e. internode length, number of stem trichomes per 0.25 mm2 surface area, and length of stem trichomes were determined both at the lower and the upper part of the shoot. Leaf characters, i.e. the length and number of trichomes per 1 mm2 on leaf margin and leaf blade, maximal length and maximal width of the leaf blade, as well as the distance between the basis of the leaf blade and its widest part were determined on well-developed leaves inserted at the lower, the middle and the upper part of the shoot. Leaf and shoot characters were measured using a binocular microscope.

After removing eleven highly correlated characters (Spearman rank coefficients: rho >  |0.9|) the following 16 characters were retained: number of stem trichomes per 0.25 mm2 surface area in the lower and upper part, the length of stem trichomes in the lower part, the number of trichomes per 1 mm2 on leaf margin of leaves inserted at the middle and the upper part of the shoot, the number of trichomes per 1 mm2 on the leaf blade of leaves inserted at the lower, the middle, and the upper part of the shoot and their lengths, the ratio between the length and width of leaves inserted at the lower, the middle, and the upper part of the shoot, and the distance between the basis of the leaf blade and its widest part of leaves inserted at the lower, the middle, and the upper part of the shoot. A dissimilarity matrix based on Bray-Curtis distances was calculated and subjected to non-metric multidimensional scaling (NMDS). Pairwise differences between groups were tested using Mantel tests applying Spearman correlation for matrix correlations. Significance was evaluated with 99,999 permutations and Bonferroni corrected.

All statistical analyses were computed in R 3.2.1. (R Development Core Team 2015). LMM and GLMM were fitted using the glmer-function implemented in the lme4-package (Bates et al. 2014). Morphometric analyses were conducted using the function metaMDS (applying KYST algorithm and using default parameter values) included in the package vegan (Oksanen et al. 2013). Mantel tests were calculated with the function metaMDS included in the package vegan (Oksanen et al. 2013).

Results

Fitness parameters did not differ between intraspecific crosses (V–V, P–P) except for V–V showing higher mean seed weight (Table 2; Fig. 1). Seed set, mean seed weight of the F1 generation as well as the germination rate of the F2 generation obtained from interspecific crosses (P–V, V–P) were intermediate to the pooled intraspecific pollination treatments (Table 2; Fig. 1). In contrast, time to the emergence of cotyledons was shorter and seedling dry weight was higher for both interspecific pollination treatments than the pooled intraspecific crosses (Table 2; Fig. 1). As the time to the emergence of cotyledons, first, second, third and fifth leaf pair were highly correlated and revealed qualitatively similar results, we show only data for cotyledons. Mortality in the F1 generation was low, as only zero to six individuals out of 82–105 germinated seeds per pollination treatment died (Table 1).

Table 2 Fixed-effect coefficients of generalized linear mixed models (GLMM; seed set, time to cotyledons, germination rate F1, germination rate F2) and linear mixed-effects models (LMM; mean seed weight, mean seedling dry weight)
Full size table
Fig. 1
figure 1

Fitness parameters of offspring derived from intra- and interspecific cross-pollinations of H. pusillum (P) und H. veselskyi (V). The first letter indicates the pollen recipient. Seed set and seed weight (mg) represent the proportion of fully developed seeds per capsule and their mean weight, respectively (a, b). Germination F1 is the percentage of F1 seeds germinated per mother plant (c). Time to cotyledons refers to the number of days from germination to the emergence of the cotyledons (d). Seedling dry weight represents the predictions of a linear mixed-effects model (LMM) based on the longest leaf measured 2 months after germination (see text for details, e). Germination rate in the F2 is the percentage of F2 seeds germinated per pollination treatment (f). Germination F2 is illustrated as barplot, as it was assessed using pooled seeds of all F1 individuals emerging from a particular pollination treatment. Boxplots show the median and the 25/75 percentiles. Whiskers are 1.5 times interquartile ranges, values outside are indicated as outliers. For the assessment of germination rate in the F2 generation seeds of all F1 individuals emerging from a particular pollination treatment were pooled

Full size image

The two-dimensional NMDS of 16 morphometric characters revealed an adequate fit (Kruskal stress value = 0.181). Clusters of intraspecific crosses were clearly separated along the first axis (Fig. 2) and significantly different according to the Mantel test (V–V vs. P–P: p < 0.001, r = 0.769). The strongly overlapping clusters of interspecific crosses occupy an intermediate position. Interspecific crosses differed from intraspecific crosses (P–P vs. P–V: p = 0.011, r = 0.224, P–P vs. V–P: p < 0.001, r = 0.442, V–V vs. P–V: p < 0.001, r = 0.287, V–V vs. V–P: p < 0.001, r = 0.402), but not from each other (P–V vs. V–P: p = 0.574, r = 0.042).

Fig. 2
figure 2

Non-metric multidimensional scaling (NMDS) of 16 morphometric characters measured on 65 individuals derived from intra- and interspecific cross-pollinations of H. pusillum (P) und H. veselskyi (V). The first letter indicates the pollen recipient. Individuals belonging to the same pollination treatment are represented by the same shading and are connected to the treatment centroid. Confidence ellipses around treatment centroids represent the standard deviation of the individuals of one treatment

Full size image

Discussion

We found no evidence of intrinsic pre- or postzygotic reproductive barriers between Heliosperma pusillum and H. veselskyi, as offspring fitness of reciprocal interspecific crosses (P–V, V–P, referred to as hybrids) was not lower than that of intraspecific crosses (P–P, V–V, referred to as parental species, Tables 1, 2). Under our experimental conditions hybrids developed faster and grew taller. This is likely connected to heterotic effects, which can appear in early-generation hybrids but often decrease in consecutive generations (Burke and Arnold 2001; Rundle and Whitlock 2001; Rhode and Cruzan 2005; Lowry et al. 2008b; Melo et al. 2014). Also in our experiment, germination rates of F2 hybrid individuals were not higher, but rather similar to those of the parental species, indicating a decrease of heterotic effects as well as the absence of recessive incompatibilities, which can appear in later generations (Rundle and Whitlock 2001; Coyne and Orr 2004). As we allowed for free pollination in F1 plants, however, we cannot exclude male sterility in F1 plants. In any event, pollination treatment and testing conditions were suitable to test for intrinsic reproductive barriers, as both parental species performed similarly well and thus did not differ in most fitness parameters except for mean seed weight, which likely represents a species-specific trait (Fig. 1; Table 2).

The absence of intrinsic reproductive barriers contrasts the scarcity of interspecific gene flow seen in genomic data (Trucchi et al. bioRxiv preprint, available at doi:10.1101/044354), suggesting that reproductive isolation is caused by extrinsic factors. Among those, spatial separation is likely of pivotal importance as populations of H. pusillum and H. veselskyi are usually separated by a forest belt extending over several hundred meters of altitude. Under such a scenario no other isolation mechanisms need to evolve, as hybrids will be rarely formed owing to spatial separation. Such distance-dependent reduction in gene flow resulting from genetically fixed ecological differences has rarely been invoked (Sobel et al. 2010); it is obviously scale-dependent, as dispersal kernels of pollen and seeds vary among plant species (Baldwin 2005). Phenology and flower morphology are unlikely causes for extrinsic prezygotic isolation as flowering times of H. pusillum and H. veselskyi overlap and both species have highly similar flowers (authors’ personal observations) likely sharing a wide range of unspecific pollinators (as previously shown for other Sileneae: Jürgens et al. 2002; Olesen and Jordano 2002; Jürgens 2006).

Alongside spatial isolation, adaptation to distinct habitat conditions can cause extrinsic reproductive isolation via pre- and postzygozic fitness disadvantages of immigrants and hybrids, respectively (Rundle and Whitlock 2001; Baack et al. 2015); the effect of ecological barriers likely increases with the extent of habitat differentiation (Ramsey et al. 2003). Reciprocal in situ transplant experiments (Rundle and Whitlock 2001; Sobel et al. 2010) of seeds and seedlings, which would provide a direct test for maladaptation of heterospecific immigrants (Rice and Hostert 1993) as well as for selection against hybrids (Rundle and Whitlock 2001; Melo et al. 2014; Baack et al. 2015) consistently failed in three consecutive years due to strong natural erosion and drought at the growing sites of H. pusillum and H. veselskyi, respectively (Bertel and Frajman, field observations). Establishment of young plants at growing sites of H. veselskyi is obviously restricted to years with particularly favourable weather conditions and ample water availability, rendering transplantation experiments infeasible (Sobel et al. 2010).

Evidence of ecologically driven reproductive isolation comes from the clear differentiation of habitats, habitat-associated functional phenotypic divergence (Bertel et al. 2016a, b), as well as heritability of morphological differentiation. The latter was confirmed by clear differentiation between the parental species grown under uniform conditions (Fig. 2), precluding that the phenotypic divergence observed in the field results solely from phenotypic plasticity. Indication of differential fitness is provided by a survey of transplanted individuals, which revealed preferential grazing of H. pusillum by herbivores at a growing site of H. veselskyi (Bertel et al. 2016a, b). This supports the recently proposed hypothesis that herbivory may act as strong selection pressure involved in reproductive isolation at early stages of speciation (Melo et al. 2014).

Despite the superior performance of F1 hybrids under our experimental conditions (Fig. 1), they may be selected against at the natural growing sites. Lower fitness of hybrids under natural versus experimental conditions (Lowry et al. 2008a) can result either from environmental stress leading to cyto-nuclear incompatibilities or—most frequently—from the differential adaptation of parental phenotypes to distinct ecological niches (Rundle and Whitlock 2001; Coyne and Orr 2004; Nosil 2012). As hybrids are morphologically intermediate (Fig. 2) independent of the crossing direction, they may perform worse in both parental habitats. The lack of habitats intermediate between humid alpine screes and dry montane rock overhangs, which might be suitable for hybrids, likely contributes to impeding the long-term persistence of hybrids with intermediate phenotypes (Choler et al. 2004).

Finally, as strength, timing and type of reproductive barriers are highly species-specific (Lowry et al. 2008a; Baack et al. 2015) further studies are urgently needed to better understand how reproductive isolation arises and finally completes plant speciation (Nosil 2012). In congruence with recent studies revealing prezygotic reproductive isolation based solely on extrinsic, ecological barriers (Westberg et al. 2010; Melo et al. 2014), our example of H. pusillum and H. veselskyi clearly shows that spatial separation of populations and adaptive phenotypic differentiation may confer a strong reduction in gene flow, which is especially significant at the onset of speciation, when intrinsic barriers are not yet in place. However, reproductive isolation solely based on environmentally, extrinsic barriers may be sensitive to environmental changes and subsequent vegetation shifts; only completion of reproductive isolation by endogenous mechanisms may on the long term ensure the stability of the diverging taxa.