Introduction

Westman (1977) first argued that to sustainably manage our global natural resources, we need to formally recognise how nature’s diverse services support human wellbeing. The benefits derived from nature were thereafter labelled as ‘ecosystem services (ES)’ (Ehrlich and Ehrlich 1981). The novelty and compelling aspect of the ES concept lie in its representation of nature as provider of services benefiting humankind. Forty years on, the concept has seen a dramatic evolution and underpins research that views ecological, economic and social systems as inextricable social-ecological systems (Anderies et al. 2004; Balvanera et al. 2017; Berkes et al. 2003; Folke 2006). The activities driving this field internationally include the MA (2005a), TEEB (Kumar 2010; ten Brink 2011) and more recently, the IPBES (2015) and the WAVES partnership ( 2017).

Ways to help support and sustain the services essential for human wellbeing are diverse, but at the onset, the concept of ES must shift from a heuristic framework to unambiguous practical applications (Polasky et al. 2011): a process which requires enactment in decision-making by policy makers and practitioners in a salient, credible and replicable manner (Daily and Matson 2008). While challenging, evidence that such a transition is under way is emerging: the concept is being formalised into systematic definitions e.g. Common International Classification of ES CICES (Haines-Young and Potschin 2013) and instruments, tools and standards are being developed (Hein et al. 2016; Maes et al. 2016; OECD 2012). As such, the intellectual community is converging towards a clarification of definitions, standards, tools and methods.

Progress in adoption of the concept in policy is also evident. The concept is being incorporated into directives at various governance levels, including international (Convention on Biological Diversity 2010), European (Bouwma et al. 2017; European Commission 2011; Maes et al. 2016) and national (DEFRA, 2012). Applying ecosystem service in decision-making is utilised as a way to quantify and qualify the ecological and socioeconomic costs and benefits of different management plans (Luck et al. 2012). Therefore, ES valuation is considered to have a high potential for application in decision-making by individuals, institutions, organisations and governments (Wallace 2008; de Groot et al. 2010). At a national level, the European Union has explicitly included the valuing of ES in their Biodiversity Strategy for 2020 (European Commission 2011). The report ties together the protection of ecosystem biodiversity with the services that it provides, making conservation management, of both, vitally important (Jax et al. 2013). As a result, many European countries are working on inputting ecosystem service assessments into their management plans, thereby influencing the implementation of ES Assessment globally (IPBES 2015). There is also evidence of application of the concept in urban planning discourses, such as in Berlin, New York, Salzburg, Seattle and Stockholm (see Hansen et al. 2015). Yet, while some progress is seen at policy levels, applications of the concept in practice are slower to materialise (Hein et al. 2016). Hansen et al. (2015) argue that the challenges of operationalization are many, including translating the ES concept into legal systems for implementation (also extensively discussed in Stępniewska et al. (2017), de Graaf et al. (2017) and Mauerhofer (2017)).

Hence, while much progress has been made in the design of procedures aiming to guide decision-making, little is known about how the knowledge of ES leads to practice impacts or to on-the-ground impacts on biodiversity, ecosystem functions, ES and the various aspects of wellbeing (Folke et al. 2002; Nahlik et al. 2012; Shanley and Lopez 2009). The knowledge of whether and how ES science translates into impacts is fragmented and disjointed. Yet, to succeed in its mission, it is essential that the science engages with the realm of its impacts.

Fundamentally mission-oriented (Cowling et al. 2008), ES science has indeed impact at its core: ‘with appropriate actions it is possible to reverse the degradation of many ES over the next 50 years, but the changes in policy and practice required are substantial and not currently underway’ (MA 2005b).Footnote 1 Underpinning this statement lies a causal assumption that for impacts to happen on the ground, impacts on policy and practice are needed. This suggests the existence of two distinct categories of impacts with a causal relationship between them: attributional impacts refer to transitory effects (impacts on policy, practice, awareness  or other direct or indirect drivers of changes) which ideally lead to consequential impacts (ultimate real on-the-ground changes on biodiversity, ecosystem functions, ES and well-being as defined within the MA).

In this paper, we focus our attention on attributional impacts as this is the category of impact most systematically reported to date (see e.g. IPBES 2015). We specifically focus on awareness, policy and practice dimensions. We define awareness as the acquisition of ‘knowledge that something exists, or [an] understanding of […] a subject […] based on information […]’ (Cambridge online dictionary). This also considers literacy, namely, the acquisition of ‘knowledge of a particular subject […]’ (Cambridge online dictionary). Policy refers to a course of action adopted or proposed at any governance level. Practice refers to the implementation of human actions and practices on the ground. Our overarching aim is to understand how ES science and associated interventions have impacted awareness, policy and practice. To address this, we answer the following three questions:

  • Has impact been defined in ES science, and if so, how?

  • What evidence exists that ES studies generate attributional impact?

  • What ES study characteristics are associated with attributional impacts?

By answering these, our intention is to contribute to breaking the broad ES impact glass ceiling: first, by highlight gaps in the coverage (definition and evidence) of impacts, and second, by identifying potential study attributes associated with attributional impacts. We conclude by proposing a systematic approach for studying and reporting the diversity of attributional as well as consequential impacts for a robust evaluation of the breadth and depth of promised transformation by ES science.

Approach

Defining Impact for the Science of Ecosystem Services

Definitions are fundamental for concerted action, as they determine the way we think and act. Here, a common understanding of ‘impact’ is fundamental to the way in which we conduct ES interventions and assess their success (let alone, that of the science). Hence, to have impact, the ES science must first define how it understands it. In this paper, we explore the ES literature to evaluate whether there exists a common and well established definition of ES impacts. We also examine how impact is defined broadly in fields as diverse as higher education, health and international development where it is set firmly as a criteria of success (incl. the World Bank, the OECD, the WHO). We believe this range to be a fair representation of impact definitions found in other, yet related, domains. Based on these findings, in the discussion, we propose a systemic definition of ES impact, a typology of impacts, and an approach for ES impact reporting which builds on the attributional and consequential impact definitions introduced above. Next, we examine whether there exist evidence that ES science generates attributional impact.

What evidence is there that the Science of Ecosystem Services generates attributional impacts?

Three databases were consulted to explore whether ES science generates attributional impacts. These were:

  • IPBES catalogue: a searchable compendium of global to subnational scales studies of biodiversity and ES (restricted access under https://www.ipbes.net/deliverables/4a-catalogue-assessments, IPBES 2015). Selected as, of the 29 ES databases reviewed in Schmidt and Seppelt (Schmidt and Seppelt 2018, see Table 1), it is the only one with a specific focus on impact.

  • Lautenbach review: a systematic review of peer-reviewed scientific ES studies assessing how current ES studies are conducted in practice (Lautenbach et al. 2019, this issue). Selected as this is the only ES study database derived from a systematic review, enabling robust statistical analyses.

  • OPERAs Exemplars: a 5-year, in-depth investigation of 13 collaboratively designed ES studies (OPERAs 2017). Selected as the database includes several impact indicators, thereby providing a sample for in-depth understanding of study characteristics related to attributional impact.

Table 1 The OPERAs Exemplar region, title and succinct study objective (see hyperlink for further details)
Full size table

Given the different nature of the databases, these are analysed independently.

IPBES catalogue (n = 102)

On 24 October 2017, the catalogue entailed 246 studies of which > 40% were randomly selected (n = 102). Within the database, each entered study can elect to self-report its impacts and associated evidence ‘[…]on policy and/or decision making, as evidenced through policy actions’ (IPBES 2015).

Lautenbach et al. review (n = 504)

The database is comprised of a random sample of peer-reviewed ES studies published in the ISI Web of Knowledge up to 31.03.2016 (n = 504). The database states whether the studies provided policy recommendations, used here as a proxy of intent for impact.

OPERAs Exemplars (n = 13)

The database comprises in-depth characteristics (namely, specific attributes) from 13 complementary ES studies (Table 1, referred to as Exemplars) and was created as part of the OPERAs EU PF7 project (http://www.operas-project.eu/). To create the database, information on a suite of Exemplar characteristics was sought through an exhaustive adaptive questionnaire (referred to online as the ‘blueprint protocol’). The questionnaire was completed three times over a 5-year period (December 2012 to November 2017) and can be freely accessed on the Oppla marketplace (https://www.oppla.eu/product/18033) for consultation and use by the wider ES community. It builds on the work of Seppelt et al. (2012) and a rigorous assessment of other published ES frameworks and protocols (see link above). It takes on average 1 hour to complete, and acted as our modus operandi for standardising the comparison, evaluation and the synthesis of these 13 Exemplars, their operationalisation and their impacts. The questionnaire engaged only the study team leads, not the broader stakeholders affected or impacted by the teams’ studies. In the questionnaire, exemplar teams were asked to self-report whether their study generated impact along the three dimensions (awareness, policy and practice) and to provide relevant evidence of such impacts.

What study characteristics are associated with attributional impacts?

Within each database, study characteristics were selected for this analysis because of their availability and their similarity across the three databases. These include the class and number of ES assessed (as well as related analytical considerations such as bundling of ES, trade-offs); stakeholders engagement attributes; geographical scale considered as well as others (see full lists in Tables 2, 3 and 4). Some characteristics unique to each database were also included if deemed particularly pertinent to generating attributional impact (e.g. scope of assessment in IPBES). We describe below the methodology used to assess associations between study characteristics and impacts within each database.

Table 2 Characteristics from the IPBES catalogue considered in the analysis. Here, five of the ten IPBES themes (labelled 1–5), were used and coded. Each theme includes between four to ten single characteristics. An additional theme was created by assessing the stated intention of each study for integration of findings into decision-making (typology of characteristics described in Daily et al. 2009). The associated variable type (categorical, ordinal or discrete) and variable label used in the results section (here italicised and in brackets, Table 8) are also presented
Full size table
Table 3 Characteristics from Lautenbach et al. (2019) considered in the analysis. The associated variable type (categorical, ordinal or discrete) are also presented
Full size table
Table 4 Characteristics from the OPERAs Exemplars’ blueprint protocol (BP) considered in the analysis. The associated variable type (categorical, ordinal or discrete) are presented, as well as the variable label used in the results section (here italicised and in brackets, Fig. 2)
Full size table

IPBES catalogue (n = 102)

For each study entry, the IPBES catalogue collates information on ten themes (subdivided into a total of 48 characteristics). Here, five of the ten IPBES themes (labelled 1–5 in Table 2) were used and coded, each including between four to ten single characteristics. An additional theme was created by assessing the stated intention of the study for integration of its findings into decision-making (typology of characteristics described in Daily et al. (2009)). Generalised linear models (GLMs) were used to test the relationship between characteristics of the studies and policy impacts. The GLMs were derived for each theme individually.

Lautenbach et al. review (n = 504)

Twenty study characteristics from the Lautenbach database are used here (Table 3) and tested against the presence or absence of specific policy recommendations. A binomial GLM with the log link (logistic regression) was used to test relationships between the response and the predictors.

OPERAs Exemplars (n = 13)

The characteristics extracted from the OPERAs Exemplars are presented in Table 4. Impacts along the three dimensions were scored as presented in Table 5.

Table 5 impact on awareness, policy impact and practice impact
Full size table

Relationships between the three impact dimensions and Exemplar characteristics were analysed by means of nonparametric pairwise correlation analysis using Kendall’s tau (Kendall 1938). The strength of the associations between numeric and categorical variables was calculated based on point biserial (two categories) or point polyserial (more than two categories) correlation coefficients (Olsson et al. 1982). The differences in correlation between each of the impact domain were visualised in a circular plot. All analyses were done in R (R Core Team 2017) using the additional package circlize (Gu et al. 2014) and polycor (Fox 2016).

Results and discussion

Defining Impact for the Science of Ecosystem Services: A typology

Discourses of impact are omnipresent in medical sciences (Greenhalgh and Fahy 2015), international development (Cameron et al. 2015; Hearn and Buffardi 2016), and education (Papastergiou 2009), yet Table 6 highlights a large variability in definitions adopted in these different fields, from comprehensive (OECD-DAC and DFID, Table 6) to narrow (impact as meeting a predefined set of objectives, World Bank, Table 6).

Table 6 Definitions of impact according to different institutions (adapted from Cameron et al. (2015), and Hearn and Buffardi (2016))
Full size table

Meanwhile, in the ES science literature, the term does not seem to have permeated widely. When impact is considered, it is narrowly defined as policy impact (IPBES 2015; Posner et al. 2016) or as effectiveness; id est the degree with which something is successful in producing a desired result (especially in the context of payments for ES). The latter definition is possibly a legacy of environmental impact assessments (EIAs) which evaluate the environmental consequences of major development projects in situ and focus on improving the quality of decisions-making to avoid unwarranted development effects. Yet, even in the EIA literature, impact and effectiveness are used either synonymously, or their distinction is tangled (compare Ervin (2003); Hockings (2003) with van Doren et al. (2013)). This confusion contributes to obscuring the potential demonstration of the breadth of impacts arising from ES science.

To achieve and demonstrate impact, it is fundamental to distinguish between effectiveness and impacts clearly. Effectiveness, as defined above, refers to a measure of success at achieving a desired result. An intervention may be effective, but if broader impacts were considered (such as trade-offs, or off-site effects), the results may not be hailed as successful overall. A more nuanced and comprehensive understanding of ES impacts is needed to encompass the diversity of effects and changes observed within the social-ecological system considered. Such a definition must also be sufficiently flexible to allow for both prospective and retrospective impact appraisals and reporting.

We therefore define ES science impact as the attributional and consequential, positive and negative, long-term effects generated through ES science (and by associated interventions) foreseen or unforeseen, intended or unintended (adapted from Independent Commission for Aid Impact 2014; OECD 2010). This definition is therefore fully inclusive of effectiveness, typically used within EIAs.

In Fig. 1, we present a framework for ES science reporting, which entails the impact definition, a typology and a reporting approach.

Fig. 1
figure 1

Systemic impact reporting framework. The figure presents (a.) a definition of impact for the Science of Ecosystem Services, (b.) a typology of impacts and (c.) a comprehensive reporting framework. (b.) Attributional impacts are precursory to consequential impacts and include transitional impacts on dimensions such as awareness, policy and practice (but could also include transitional effects on other direct or indirect drivers of change, e.g. technological developments). Attributional impacts can occur without consequential impacts. Consequential impacts are ultimately, the real, on-the-ground impacts on dimensions such as biodiversity, ecosystem functions, ecosystem services and well-being. (c.) We exemplify how this framework (adapted from Hearn and Buffardi 2016) can be used in practice as a reporting, thinking or impact mapping tool using a hypothetical reforestation intervention under a PES scheme

Full size image

The typology characterises the different dimensions of ES science impacts, and thereby contributes towards a consistent and systematic reporting approach. A distinction between attributional and consequential impacts is fundamental, as the former is only transitory, and does not capture on-the-ground impacts (desired or not). An ES intervention may impact policy yet never realise the desired effects on biodiversity, ES, ecosystem functions or human-wellbeing. Attributional impacts (notably impacts on policy) remain nevertheless the category of impacts most systematically reported to date (see IPBES (2015) and Schmidt and Seppelt (2018) for their description of 29 ES databases).

The definition and the systemic typology further differentiates between what are seen as positive or negative impacts, as well as foreseen, unforeseen, intended or unintended effects. Using a hypothetical reforestation intervention under a PES scheme, in Fig. 1, we exemplify how this framework can be used in practice as a reporting, thinking or impact mapping tool. This framework for understanding ES impacts and their reporting offers clear boundaries to categorise the depth with which impact is considered and reported.

What evidence exists that ES studies generate attributional impact? Towards systematic reporting

The lack of definition and consensus as to how impact is defined has undoubtedly contributed to the shortage of concerted effort to report impact, and the accompanying limited evidence that the ES science generates impacts. The extent with which ES science has generated attributional impacts is thin. Within the IPBES catalogue, we found that only 26 (25%) of the 102 randomly selected studies self-reported policy impact. Seven (7%) were independently reviewed. Within the Lautenbach database, the results are similar: only 33% of studies provide some policy recommendations. The findings from the Exemplars contrast, however, with those above: all Exemplars reported impact along at least one of the three dimensions (examples of which are presented in Table 7) and 62% believe having had some impact on policy. The systematic reporting imposed by the adaptive questionnaire in the OPERAs Exemplars (which also supported the study design) may have contributed to such attainment.

Table 7 Evidence of Exemplar impacts on awareness, policy and practice as collated using the blueprint protocol
Full size table

Until impact becomes systematically reported, the evidence that ES concept does generate impact may remain thin. A few simple steps could go a long way in addressing this dearth. Impact being core to ES science, it would seem reasonable that peer-reviewed journals in the field request short statements of impact (intended and/or realised), alongside abstracts and keywords. This would promote the importance of thoughtful impact planning and reporting. The use of protocols for study design and reporting (such as the blueprint protocol questionnaire briefly introduced here) could also be considered as a tool to remind teams of the importance of impact planning and reporting. Such protocols serve as vehicles to summarise outcomes from a diverse set of studies, but also, as a thinking tool to integrate specific study considerations (including impact). Finally, global efforts at creating compendia of ES studies need to capture ES science impacts more broadly and consistently. Adopting a comprehensive definition and reporting framework such as the one presented here is essential to fully capture the breadth and depth of ES science impacts.

What study characteristics are associated with impacts? Designing studies for impact

Planning for impact requires an understanding of drivers of impact. Within each database, we explored which study characteristics were associated with different dimensions of attributional impacts. In IPBES and Lautenbach, policy impacts alone were considered. In OPERAs Exemplars, the three dimensions of awareness, policy and practice impacts were analysed.

IPBES catalogue (n = 102)

The results from the GLMs are presented in Table 8. Studies with policy impact show a significantly broader coverage of characteristics for all ordinal characteristics (sign. levels *** column B). However, they did not cover a significantly different selection of ES, but differed in the scales addressed: in these studies, site (*) and subnational scales ( ) were significantly more often studied. Furthermore, they are characterised by stakeholder engagement (**), and engage significantly more with affected people (*); people who actively manage the ecosystems (*), and policy makers ( ). They also have an increased emphasis on drivers of change of ES (**) and explicitly address the role of biodiversity (*). Finally, they differ methodologically by focusing on the ecosystem themselves (*) and on the valuation of the ES ().

Table 8 Factors characterising the IPBES studies with policy impact. Significant characteristics are indicated in italics (p < 0.1) and with subscripts ***p < 0.001; **p < 0.01; *p < 0.05. [...] indicates a list of further individual ES, which were tested, but not explicitly listed (not significant). Table 2 provides further descriptions of these characteristics (and their categorisation within the five IPBES themes)
Full size table

Lautenbach et al. review (n = 504)

Studies providing policy recommendations were significantly associated with a number of characteristics (Table 9). Stakeholder involvement mattered: a higher frequency of recommendations was observed in studies where stakeholders were involved. Studies involving distant beneficiaries and experts or that involved stakeholders in a consultation role provided significantly more recommendations than others. Studies that assessed trade-offs by map overlay also provided significantly more recommendations than others—this is likely related to the effect of mapping ES at all: studies that mapped ES reported significantly more recommendations. Studies that assessed bundles of ES reported more specific recommendations than others; however, the case number was low and the effect only marginally significant. Significantly higher frequencies of specific recommendations were also observed for studies that considered policy instruments (low number of cases), that used models different from lookup tables and combined demand and supply side analysis.

Table 9 Factors charaterising Lautenbach et al. (2019) studies with policy recommendations (proxy for impact). Regression coefficients are reported at the link scale of the logistic regression, significant characteristics are indicated with subscripts ***p < 0.001; **p < 0.01; *p < 0.05; p < 0.1
Full size table

OPERAs Exemplars (n = 13)

The three dimensions of impact assessed were correlated across the 13 Exemplars. A moderately strong positive correlation was identified between impact on awareness and impact on practice (τ = 0.56) and between impact on policy and impact on practice (τ = 0.44) but not between impact on awareness and policy (τ = 0.06: policy makers are possibly sufficiently aware of the ES concept, such that mainly practitioners and stakeholders other than policy makers achieve greater awareness through ES engagement).

Associations between Exemplar characteristics and impact were manifold: studies that addressed a more specific problem were positively associated with a larger impact on policy (τ = 0.45) and with impact on awareness (τ = 0.20). Studies that took place at a smaller scale were positively associated with impact on awareness (τ = 0.35). For impact on policy, the impact increased with scale (τ = 0.23). Studies that scored higher with respect to robustness of the analysis were positively associated with impact on all three dimensions: τ = 0.59, 0.50 and 0.57 for impact on awareness, policy and practice respectively. The number of ES categories addressed in an Exemplar was associated with the impact on awareness (0.45).

The indicator measuring the different aspects of stakeholder involvement was positively associated with impact on awareness and impact on practice (τ =  0.42 and 0.39, respectively). The diversity of approaches used to engage with stakeholders was positively associated with impact on practice (τ = 0.49). The diversity of the stakeholders involved was positively associated with impact on awareness (τ = 0.49). The number of stakeholders involved was positively associated with impact on awareness (τ = 0.23) and impact on practice (τ = 0.37).

Exemplars that addressed the effects of policy instruments directly instead of indirectly had a higher impact in all dimensions: the biserial correlation coefficients were r = − 0.78 for impact on policy, r = − 0.44 for impact on awareness and r = − 0.52 for impact on practice—Exemplars that did not address the effects of policy instruments directly had on average a lower impact in those categories compared to those that did. Interventional studies were associated with a higher impact on policy compared to observational studies (polyserial correlation coefficient r = 0.54) but a lower impact in practice (polyserial correlation coefficient r = − 0.46).

Despite relatively low correlation coefficients (between 0.2 and 0.5) overall, the relative importance of single factors becomes obvious from Fig. 2. In this figure, the upper half of the circles represents the different types of impacts and the lower half, the characteristics of the studies. The width of each connection indicates the strength of the Kendall’s correlation between study characteristic and impact (negative correlations are indicated with dashed outline). Scientific robustness shows the highest positive correlation with all types of impacts: awareness, policy and practice. Additional important factors include the number of ES investigated, the specificity of the study, the inclusion of stakeholders at various study stages, the degree of organisation of the stakeholders, the diversity of stakeholders included and the scale of the study. However, the correlation coefficients are in sum not as high; they vary strongly between the impact of awareness, policy and practice, and they show in parts negative correlation to single types of impacts. One illustrative example is, for instance, scale where larger scale is negatively correlated to impact on practice. Whether or not a study includes stakeholders of opposing views shows least correlation with any of the impacts. Similarly, the diversity of approaches to address stakeholders is not highly correlated with impacts.

Fig. 2
figure 2

Comprehensive graphical overview of associations between Exemplar characteristics and attributional impact dimensions. The upper half of the circle represents the different types of impacts and the lower half, the characteristics of the studies. Only the characteristics which are statistically significant are included here. The width of each connection indicates the strength of the Kendall’s correlation between study characteristic and impact. Negative correlations are indicated with dashed outline. Specificness refers to specificity of study. Sc, Instr, Sth, Div, Var and Opp, abbreviations stand for scientific, instruments, stakeholders, diversity, various and opposing, respectively (see Table 4 for corresponding characteristics)

Full size image

ES study characteristics associated with impacts: synthesis from the three databases

Table 10 presents a summary of results from the IPBES catalogue, the Lautenbach review and the OPERAs Exemplars analyses. For the OPERAs Exemplars, correlations greater or equal to 0.35 are included (* indicates negative correlation). For the Lautenbach review and the IPBES catalogue, significant characteristics and predictors of studies with recommendations/impacts are included (p < 0.1). Characteristics in italics are common to all databases.

Table 10 Study characteristics associated with policy impacts (IPBES), recommendations (systematic review) and impact dimensions (Exemplars). For the systematic review and IPBES catalogue, only significant characteristics (predictors of studies with recommendations/impacts) are included (p < 0.1). For the Exemplars analysis, only correlations greater or equal to 0.35 are included. Characteristics in italics are common to all databases
Full size table

While there does not exist a single sweeping intervention or study design affecting consistently all dimensions of attributional impact, our findings highlight four characteristics which appear more frequently (see rows with three Xs, Table 10). These are study robustness, integration of policy instruments into study design, stakeholder involvement and type of stakeholders involved. One could argue that these are the minimum baseline characteristics for impact. Our findings further suggest that to achieve impact along specific dimensions, studies must have a bespoke design: the characteristics associated with impacts on awareness, policy and practice were shown to differ. As a corollary, planning for impact matters. If impact on practice is desired, then a study conducted at smaller scale (e.g. local) which engages with many stakeholders may be more effective than a global modelling exercise testing different policy options and instruments.

The questionnaire can aid in structuring assessments, studies and monitoring programs for impact, by providing various elements for considerations. By highlighting topics such as the analysis of uncertainties, offsite effects or the effects of different policy instruments, it helps researchers track the requirements for, and think early about, desired impacts.

Conclusions

A few lessons can be learned from this study, from which we propose a suite of recommendations.

Lesson-learned 1: There are no clear agreed definitions of impacts in ES science. Such lack of consensus in (and confusion between) definitions of effectiveness, impacts and other measures of a study or intervention’s effects contributes to obscuring the potential demonstration of outcomes arising from ES science. Recommendations 1: given the diversity of ES, functions, associated intervention and studies as well as the breadth of potential impacts, impact ought to be defined broadly as the attributional and consequential, positive and negative, long-term effects generated through ES science (and by associated interventions) foreseen or unforeseen, intended or unintended. A recognition of the different categories of impacts is also important. Attributional impacts (impacts on policy, practice, awareness or other direct/indirect drivers) refer to transitory effects which ideally, but not necessarily, lead to consequential impacts (real, on the ground effects on biodiversity, ES, ecosystem functions and wellbeing).

Lesson-learned 2: There is no broad and systematic collection of ES science impact. This leads to a limited body of evidence that ES studies or interventions yield impact. Even the IPBES catalogue does not collate broad or diverse evidence of impact beyond policy, let alone consequential impacts. Recommendations 2: systematic reporting is critically needed for both attributional and consequential impacts. With minimal efforts, the existing compendia (e.g. IPBES) could be expanded to include broader, systemic and comprehensive impact reporting. The impact framework (definition, typology and reporting framework) presented in Fig. 1 offers such a flexible framework, and clarifies the depth with which impact is considered. It considers foreseen and unforeseen effects, but also allows for both prospective and retrospective reporting. In parallel, expected or realised impacts statement should be presented alongside abstracts and keywords, in ES focused peer-reviewed journals. Finally, the use of well grounded reporting frameworks, such as the questionnaire (blueprint protocol) introduced here, can help teams engage more with the realm of impact.

Lesson-learned 3: Across the three databases considered here (IPBES, Lautenbach review and OPERAs Exemplars), a few key study characteristics were found to be more frequently associated with attributional impacts: these are study robustness, integration of policy instruments into study design, stakeholder involvement and type of stakeholders involved. We see these as the minimum baseline for impact. Meanwhile, the 5-year investigation of the OPERAs Exemplars enabled greater in-depth investigation of associations between study characteristics and specific dimensions of attributional impact. While many characteristics were associated with impacts on awareness and practice, the characteristics linked with impact on policy tended to be distinct. Recommendations 3: designing studies for robustness and reliability is fundamental to all dimensions of attributional impact—study robustness was consistently associated with impact on awareness, policy and practice. Meaningful engagement with stakeholders (see Durham et al. 2014 for guidelines) and careful selection of stakeholder types was also consistently associated with attributional impact. To effect specific dimensions of impacts however, studies must have a bespoke design. Studies conducted at smaller scale (e.g. local) and engaging with many stakeholders were shown to be more effective at generating impact on practice than global modelling studies testing policy options.

In this paper, we have demonstrated that there currently exists no clear consensus on a definition of ES science impacts. We have also shown that limited information is collated on attributional impacts (let alone, consequential impacts). Through robust statistical analyses of three extensive databases of ES studies, we show that certain study characteristics are a key for achieving attributional impact (incl. study robustness and stakeholder engagement).

To break the ecosystem services impact glass ceiling; however, a concerted, global and systematic effort to define, assess and collate evidence of both attributional and consequential impacts is urgently needed.