Breaking the Ecosystem Services Glass Ceiling : Realising Impact

Through changes in policy and practice, the inherent intent of the ecosystem services (ES) concept is to safeguard ecosystems for human wellbeing. While impact is intrinsic to the concept, little is known about how and whether ES science leads to impact. Evidence of impact is needed. Given the lack of consensus on what constitutes impact, we differentiate between attributional impacts (transitional impacts on policy, practice, awareness or other drivers) and consequential impacts (real, on-the-ground impacts on biodiversity, ES, ecosystem functions and human wellbeing) impacts. We conduct rigorous statistical analyses on three extensive databases for evidence of attributional impact (the form most prevalently reported): the IPBES catalogue (n = 102), the Lautenbach systematic review (n = 504) and a 5-year in-depth survey of the OPERAs Exemplars (n = 13). To understand the drivers of impacts, we statistically analyse associations between study characteristics and impacts. Our findings show that there exists much confusion with regard to defining ES science impacts, and that evidence of attributional impact is scarce: only 25% of the IPBES assessments self-reported impact (7% with evidence); in our meta-analysis of Lautenbach’s systematic review, 33% of studies provided recommendations indicating intent of impacts. Systematic impact reporting was imposed by design on the OPERAs Exemplars: 100% reported impacts, suggesting the importance of formal impact reporting. The generalised linear models and correlations between study characteristics and attributional impact dimensions highlight four characteristics as minimum baseline for impact: study robustness, integration of policy instruments into study design, stakeholder involvement and type of stakeholders involved. Further in depth examination of the OPERAs Exemplars showed that study characteristics associated with impact on awareness and practice differ from those associated with impact on policy: to achieve impact along specific dimensions, bespoke study designs are recommended. These results inform targeted recommendations for ES science to break its impact glass ceiling.


Introduction
first argued that to sustainably manage our global natural resources, we need to formally recognise how nature's diverse services support human wellbeing. The benefits derived from nature were thereafter labelled as 'ecosystem services (ES)' (Ehrlich and Ehrlich 1981). The novelty and compelling aspect of the ES concept lie in its representation of nature as provider of services benefiting humankind. Forty years on, the concept has seen a dramatic evolution and underpins research that views ecological, economic and social systems as inextricable social-ecological systems (Anderies et al. 2004;Balvanera et al. 2017;Berkes et al. 2003;Folke 2006). The activities driving this field internationally include the MA (2005a), TEEB (Kumar 2010;ten Brink 2011) and more recently, the IPBES (2015) and the WAVES partnership ( 2017).
Ways to help support and sustain the services essential for human wellbeing are diverse, but at the onset, the concept of ES must shift from a heuristic framework to unambiguous practical applications (Polasky et al. 2011): a process which requires enactment in decision-making by policy makers and practitioners in a salient, credible and replicable manner (Daily and Matson 2008). While challenging, evidence that such a transition is under way is emerging: the concept is being formalised into systematic definitions e.g. Common International Classification of ES CICES (Haines-Young and Potschin 2013) and instruments, tools and standards are being developed (Hein et al. 2016;Maes et al. 2016;OECD 2012). As such, the intellectual community is converging towards a clarification of definitions, standards, tools and methods.
Progress in adoption of the concept in policy is also evident. The concept is being incorporated into directives at various governance levels, including international (Convention on Biological Diversity 2010), European (Bouwma et al. 2017;European Commission 2011;Maes et al. 2016) and national (DEFRA, 2012). Applying ecosystem service in decision-making is utilised as a way to quantify and qualify the ecological and socioeconomic costs and benefits of different management plans (Luck et al. 2012). Therefore, ES valuation is considered to have a high potential for application in decision-making by individuals, institutions, organisations and governments (Wallace 2008;de Groot et al. 2010). At a national level, the European Union has explicitly included the valuing of ES in their Biodiversity Strategy for 2020 (European Commission 2011). The report ties together the protection of ecosystem biodiversity with the services that it provides, making conservation management, of both, vitally important (Jax et al. 2013). As a result, many European countries are working on inputting ecosystem service assessments into their management plans, thereby influencing the implementation of ES Assessment globally (IPBES 2015). There is also evidence of application of the concept in urban planning discourses, such as in Berlin, New York, Salzburg, Seattle and Stockholm (see Hansen et al. 2015). Yet, while some progress is seen at policy levels, applications of the concept in practice are slower to materialise (Hein et al. 2016). Hansen et al. (2015) argue that the challenges of operationalization are many, including translating the ES concept into legal systems for implementation (also extensively discussed in Stępniewska et al. (2017), de Graaf et al. (2017 and Mauerhofer (2017)).
Hence, while much progress has been made in the design of procedures aiming to guide decision-making, little is known about how the knowledge of ES leads to practice impacts or to on-the-ground impacts on biodiversity, ecosystem functions, ES and the various aspects of wellbeing (Folke et al. 2002;Nahlik et al. 2012;Shanley and Lopez 2009). The knowledge of whether and how ES science translates into impacts is fragmented and disjointed. Yet, to succeed in its mission, it is essential that the science engages with the realm of its impacts.
Fundamentally mission-oriented (Cowling et al. 2008), ES science has indeed impact at its core: 'with appropriate actions it is possible to reverse the degradation of many ES over the next 50 years, but the changes in policy and practice required are substantial and not currently underway' (MA 2005b). 1 Underpinning this statement lies a causal assumption that for impacts to happen on the ground, impacts on policy and practice are needed. This suggests the existence of two distinct categories of impacts with a causal relationship between them: attributional impacts refer to transitory effects (impacts on policy, practice, awareness or other direct or indirect drivers of changes) which ideally lead to consequential impacts (ultimate real on-the-ground changes on biodiversity, ecosystem functions, ES and well-being as defined within the MA).
In this paper, we focus our attention on attributional impacts as this is the category of impact most systematically reported to date (see e.g. IPBES 2015). We specifically focus on awareness, policy and practice dimensions. We define awareness as the acquisition of 'knowledge that something exists, or [an] understanding of […] a subject […] based on information […]' (Cambridge online dictionary). This also considers literacy, namely, the acquisition of 'knowledge of a particular subject […]' (Cambridge online dictionary). Policy refers to a course of action adopted or proposed at any governance level. Practice refers to the implementation of human actions and practices on the ground. Our overarching aim is to understand how ES science and associated interventions have impacted awareness, policy and practice. To address this, we answer the following three questions: By answering these, our intention is to contribute to breaking the broad ES impact glass ceiling: first, by highlight gaps in the coverage (definition and evidence) of impacts, and second, by identifying potential study attributes associated with attributional impacts. We conclude by proposing a systematic approach for studying and reporting the diversity of attributional as well as consequential impacts for a robust evaluation of the breadth and depth of promised transformation by ES science.

Approach
Defining Impact for the Science of Ecosystem Services Definitions are fundamental for concerted action, as they determine the way we think and act. Here, a common understanding of 'impact' is fundamental to the way in which we conduct ES interventions and assess their success (let alone, that of the science). Hence, to have impact, the ES science must first define how it understands it. In this paper, we explore the ES literature to evaluate whether there exists a common and well established definition of ES impacts. We also examine how impact is defined broadly in fields as diverse as higher education, health and international development where it is set firmly as a criteria of success (incl. the World Bank, the OECD, the WHO). We believe this range to be a fair representation of impact definitions found in other, yet related, domains. Based on these findings, in the discussion, we propose a systemic definition of ES impact, a typology of impacts, and an approach for ES impact reporting which builds on the attributional and consequential impact definitions introduced above. Next, we examine whether there exist evidence that ES science generates attributional impact.
What evidence is there that the Science of Ecosystem Services generates attributional impacts? Given the different nature of the databases, these are analysed independently.

IPBES catalogue (n = 102)
On 24 October 2017, the catalogue entailed 246 studies of which > 40% were randomly selected (n = 102). Within the database, each entered study can elect to self-report its impacts and associated evidence '[…]on policy and/or decision making, as evidenced through policy actions' (IPBES 2015).

Lautenbach et al. review (n = 504)
The database is comprised of a random sample of peerreviewed ES studies published in the ISI Web of Knowledge up to 31.03.2016 (n = 504). The database states whether the studies provided policy recommendations, used here as a proxy of intent for impact.

OPERAs Exemplars (n = 13)
The database comprises in-depth characteristics (namely, specific attributes) from 13 complementary ES studies (Table 1, referred to as Exemplars) and was created as part of the OPERAs EU PF7 project (http://www.operas-project.eu/). To create the database, information on a suite of Exemplar characteristics was sought through an exhaustive adaptive questionnaire (referred to online as the 'blueprint protocol'). The questionnaire was completed three times over a 5-year period (December 2012 to November 2017) and can be freely accessed on the Oppla marketplace (https://www.oppla.eu/ product/18033) for consultation and use by the wider ES community. It builds on the work of Seppelt et al. (2012) and a rigorous assessment of other published ES frameworks and protocols (see link above). It takes on average 1 hour to complete, and acted as our modus operandi for standardising the comparison, evaluation and the synthesis of these 13 Exemplars, their operationalisation and their impacts. The questionnaire engaged only the study team leads, not the broader stakeholders affected or impacted by the teams' studies. In the questionnaire, exemplar teams were asked to self-report whether their study generated impact along the three dimensions (awareness, policy and practice) and to provide relevant evidence of such impacts.
What study characteristics are associated with attributional impacts?
Within each database, study characteristics were selected for this analysis because of their availability and their similarity across the three databases. These include the class and number of ES assessed (as well as related analytical considerations such as bundling of ES, trade-offs); stakeholders engagement attributes; geographical scale considered as well as others (see full lists in Tables 2, 3 and 4). Some characteristics unique to each database were also included if deemed particularly pertinent to generating attributional impact (e.g. scope of assessment in IPBES). We describe below the methodology used to assess associations between study characteristics and impacts within each database.

IPBES catalogue (n = 102)
For each study entry, the IPBES catalogue collates information on ten themes (subdivided into a total of 48 characteristics). Here, five of the ten IPBES themes (labelled 1-5 in Table 2) were used and coded, each including between four to ten single characteristics. An additional theme was created by assessing the stated intention of the study for integration of its findings into decisionmaking (typology of characteristics described in Daily et al. (2009)). Generalised linear models (GLMs) were used to test the relationship between characteristics of the studies and policy impacts. The GLMs were derived for each theme individually.

Lautenbach et al. review (n = 504)
Twenty study characteristics from the Lautenbach database are used here (Table 3) and tested against the presence or absence of specific policy recommendations. A binomial GLM with the log link (logistic regression) was used to test relationships between the response and the predictors.

OPERAs Exemplars (n = 13)
The characteristics extracted from the OPERAs Exemplars are presented in Table 4. Impacts along the three dimensions were scored as presented in Table 5.
Relationships between the three impact dimensions and Exemplar characteristics were analysed by means of nonparametric pairwise correlation analysis using Kendall's tau Blue carbon in the Balearic Islands: the future of seagrass To assess the co-beneficiary management of seagrass ecosystems for blue carbon in the Balearic Islands to develop strategies for mitigation of CO 2 emissions through conservation of coastal marine ecosystems Barcelona Desert in the city: urban dunes in Barcelona To learn to construct and maintain dunes on urban beaches to optimise the flows of ecosystem services such as protection against sea level rise, and to learn how to shape social attitudes to make intensive recreational use of beaches compatible with the protection of the dunes Lower Danube Traversing waters: waterway management in the lower Danube To identify and raise awareness of the societal, economic and environmental values of wetlands, and to explore the relationship between restored and sustainably managed wetlands and socio-economic welfare to inform decision-making in the Danube river basin Dublin Cultural seascapes: sociocultural benefits of the Irish coastline To research the expression of cultural ecosystem service values in a coastal setting, and to consider the contribution of ecosystem services approaches to consultation within land use planning Europe Pan European policy: conflicts and synergies across Europe To evaluate how recent and forthcoming EU policy developments affect the levels of ecosystem services and natural capital in Europe French Alps Land use legacies: looking at infrastructure development in the central Alps To analyse future land use trajectories and their effects on networks of biodiversity and ecosystem services in the Grenoble urban area, in order to inform territorial planning and management Global Tools for today: policy and market instruments for climate change mitigation and habitat conservation To use the ecosystem services concept to identify and communicate geographic areas and management solutions that support the multiple goals of biodiversity conservation, climate change mitigation and feeding an increasing global population Scottish wetlands Inner Forth: cultural value mapping along the Inner Forth To understand how society, policy makers and landowners view and vision the Inner Forth river system, and how they might choose to adapt to climate change Peru Ecosystem services and climate strategies: analysis of trade-offs in Peru with pantropical comparisons To support local, regional and national decision-making on territorial management and ecosystem services in Peru, through stakeholder involvement and research on trade-offs between ecosystem services Portugal Montado More than Cork: cultural landscapes in the Montado To employ the ecosystem services and natural capital concepts to combine the productive, ecological and cultural aspects of socio-ecological systems to promote improved management of cork trees and help facilitate the wellbeing of the Montado for generations to come. Scottish Pentland Hills From highlands to lowlands: applying environmental policy in Scotland at all levels To match the needs of land use management and biodiversity policy in Scotland by contributing to the science, information and assessment methods necessary to support policy implementation Swiss Alps All eyes on the future: transition pathways and sustainability in the Swiss Alps To answer the question: which policy strategies can balance the supply of and demand for mountain ecosystem services in the future? Wine producing regions Values and vines: reaching out to consumers on responsible wines To understand how different players in the wine value chain (producers, retailers, consumers) influence wine production, and thus the ecosystem services provided by vineyard ecosystems and to promote more sustainable vineyard management to increase ecosystem services (Kendall 1938). The strength of the associations between numeric and categorical variables was calculated based on point biserial (two categories) or point polyserial (more than two categories) correlation coefficients (Olsson et al. 1982). The differences in correlation between each of the impact domain were visualised in a circular plot. All analyses were done in R (R Core Team 2017) using the additional package circlize (Gu et al. 2014) and polycor (Fox 2016).

Results and discussion
Defining Impact for the Science of Ecosystem Services: A typology Discourses of impact are omnipresent in medical sciences (Greenhalgh and Fahy 2015), international development (Cameron et al. 2015;Hearn and Buffardi 2016), and Table 2 Characteristics from the IPBES catalogue considered in the analysis. Here, five of the ten IPBES themes (labelled 1-5), were used and coded. Each theme includes between four to ten single characteristics. An additional theme was created by assessing the stated intention of each study for integration of findings into decision-making (typology of characteristics described in Daily et al. 2009). The associated variable type (categorical, ordinal or discrete) and variable label used in the results section (here italicised and in brackets,   (Papastergiou 2009), yet Table 6 highlights a large variability in definitions adopted in these different fields, from comprehensive (OECD-DAC and DFID, Table 6) to narrow (impact as meeting a predefined set of objectives, World Bank, Table 6). Meanwhile, in the ES science literature, the term does not seem to have permeated widely. When impact is considered, it is narrowly defined as policy impact (IPBES 2015;Posner et al. 2016) or as effectiveness; id est the degree with which something is successful in producing a desired result (especially in the context of payments for ES). The latter definition is possibly a legacy of environmental impact assessments (EIAs) which evaluate the environmental consequences of major development projects in situ and focus on improving the quality of decisions-making to avoid unwarranted development effects. Yet, even in the EIA literature, impact and effectiveness are used either synonymously, or their distinction is tangled (compare Ervin (2003);Hockings (2003) with van Doren et al. (2013)). This confusion contributes to obscuring the potential demonstration of the breadth of impacts arising from ES science.
To achieve and demonstrate impact, it is fundamental to distinguish between effectiveness and impacts clearly. Effectiveness, as defined above, refers to a measure of success at achieving a desired result. An intervention may be effective,  (0) Ordinal: impact on policy: high (4) to none (0) Ordinal: impact on practice: high (4) to none (0) *E.g. experts, local beneficiaries, decision makers, organisations (NGOs, industry). See blueprint protocol for complete list. **Incl. exploring techniques (e.g. mind mapping, problem tree analysis, SWOT analysis, timelines), closing down and deciding techniques (e.g. voting, ranking, prioritisation, multicriteria decision modelling), other (e.g. one-to-one meetings, interviews, questionnaires, surveys, knowledge exchange groups, workshops, talks, practical demonstrations) a Study robustness was assessed following the classification proposed by Mupepele et al. (2016) which discriminates between seven levels. At the highest level, robust evidence is generated through systematic reviews. At the lowest are studies without underlying data b The specificity of the question addressed was coded from the stated purpose and rationale of the analysis. A study addressing a specific management question was coded as 'highly specific' (Barcelona , Table 1, stabilisation of localised sand dunes) while more abstract purposes were coded as vague (Dublin, Table 1, supporting land planning by assessing socio-cultural valuation). We hypothesised that a more specific question would lead to a higher impact on practice and policy since the costs and benefits of ES management would be more direct and easier to communicate c The scale at which the ES study had been conducted included five categories from local scale to global scale. We hypothesised that smaller scale assessments would have a higher impact on practice since costs and benefits are easier to oversee at smaller scales (likewise with the timing of land use transitions) d Exemplars were categorised according to how they considered policy instruments: directly or indirectly. The former includes Exemplars that study policy instruments or their effects explicitly, the latter, those that consider policy instruments, but only marginally or indirectly e The BiodivERsA Stakeholder Engagement standard (Durham et al. 2014) informed our selection of stakeholder engagement characteristics listed above but if broader impacts were considered (such as trade-offs, or off-site effects), the results may not be hailed as successful overall. A more nuanced and comprehensive understanding of ES impacts is needed to encompass the diversity of effects and changes observed within the social-ecological system considered. Such a definition must also be sufficiently flexible to allow for both prospective and retrospective impact appraisals and reporting. We therefore define ES science impact as the attributional and consequential, positive and negative, long-term effects generated through ES science (and by associated interventions) foreseen or unforeseen, intended or unintended (adapted from Independent Commission for Aid Impact 2014; OECD 2010). This definition is therefore fully inclusive of effectiveness, typically used within EIAs.
In Fig. 1, we present a framework for ES science reporting, which entails the impact definition, a typology and a reporting approach.
The typology characterises the different dimensions of ES science impacts, and thereby contributes towards a consistent and systematic reporting approach. A distinction between attributional and consequential impacts is fundamental, as the former is only transitory, and does not capture on-the-ground impacts (desired or not). An ES intervention may impact policy yet never realise the desired effects on biodiversity, ES, ecosystem functions or human-wellbeing. Attributional impacts (notably impacts on policy) remain nevertheless the category of impacts most systematically reported to date (see IPBES (2015) and Schmidt and Seppelt (2018) for their description of 29 ES databases).
The definition and the systemic typology further differentiates between what are seen as positive or negative impacts, as well as foreseen, unforeseen, intended or unintended effects. Using a hypothetical reforestation intervention under a PES scheme, in Fig. 1, we exemplify how this framework can be used in practice as a reporting, thinking or impact mapping tool. This framework for understanding ES impacts and their reporting offers clear boundaries to categorise the depth with which impact is considered and reported.

What evidence exists that ES studies generate attributional impact? Towards systematic reporting
The lack of definition and consensus as to how impact is defined has undoubtedly contributed to the shortage of concerted effort to report impact, and the accompanying limited evidence that the ES science generates impacts. The extent with which ES science has generated attributional impacts is thin. Within the IPBES catalogue, we found that only 26 (25%) of the 102 randomly selected studies self-reported policy impact. Seven (7%) were independently reviewed. Within the Lautenbach database, the results are similar: only 33% of studies provide some policy recommendations. The findings from the Exemplars contrast, however, with those above: all Exemplars reported impact along at least one of the three dimensions (examples of which are presented in Table 7) and 62% believe having had some impact on policy. The systematic reporting imposed by the adaptive questionnaire in the OPERAs Exemplars (which also supported the study design) may have contributed to such attainment.
Until impact becomes systematically reported, the evidence that ES concept does generate impact may remain thin. A few simple steps could go a long way in addressing this dearth. Impact being core to ES science, it would seem reasonable that  Cameron et al. (2015), and Hearn and Buffardi (2016)) Institution

Definition of impact European Commission (EC)
In an impact assessment process, the term impact describes all the changes which are expected to happen due to the implementation and application of a given policy option/intervention. Such impacts may occur over different timescales, affect different actors and be relevant at different scales (local, regional, national and EU). In an evaluation context, impact refers to the changes associated with a particular intervention which occur over the longer term (   peer-reviewed journals in the field request short statements of impact (intended and/or realised), alongside abstracts and keywords. This would promote the importance of thoughtful impact planning and reporting. The use of protocols for study design and reporting (such as the blueprint protocol questionnaire briefly introduced here) could also be considered as a tool to remind teams of the importance of impact planning and reporting. Such protocols serve as vehicles to summarise outcomes from a diverse set of studies, but also, as a thinking tool to integrate specific study considerations (including impact). Finally, global efforts at creating compendia of ES studies need to capture ES science impacts more broadly and consistently. Adopting a comprehensive definition and reporting framework such as the one presented here is essential to fully capture the breadth and depth of ES science impacts.
What study characteristics are associated with impacts? Designing studies for impact Planning for impact requires an understanding of drivers of impact. Within each database, we explored which study characteristics were associated with different dimensions of attributional impacts. In IPBES and Lautenbach, policy impacts alone were considered. In OPERAs Exemplars, the three dimensions of awareness, policy and practice impacts were analysed.   The results from the GLMs are presented in Table 8. Studies with policy impact show a significantly broader coverage of characteristics for all ordinal characteristics (sign. levels *** column B). However, they did not cover a significantly different selection of ES, but differed in the scales addressed: in these studies, site (*) and subnational scales ( ) were significantly more often studied. Furthermore, they are characterised by stakeholder engagement (**), and engage significantly more with affected people (*); people who actively manage the ecosystems (*), and policy makers ( ). They also have an increased emphasis on drivers of change of ES (**) and explicitly address the role of biodiversity (*). Finally, they differ methodologically by focusing on the ecosystem themselves (*) and on the valuation of the ES ().

Lautenbach et al. review (n = 504)
Studies providing policy recommendations were significantly associated with a number of characteristics (Table 9). Stakeholder involvement mattered: a higher frequency of recommendations was observed in studies where stakeholders were involved. Studies involving distant beneficiaries and experts or that involved stakeholders in a consultation role provided significantly more recommendations than others. Studies that assessed trade-offs by map overlay also provided significantly more recommendations than others-this is likely related to the effect of mapping ES at all: studies that mapped ES reported significantly more recommendations. Studies that assessed bundles of ES reported more specific recommendations than others; however, the case number was low and the effect only marginally significant. Significantly higher frequencies of specific recommendations were also observed for studies that considered policy instruments (low number of cases), that used models different from lookup tables and combined demand and supply side analysis.

OPERAs Exemplars (n = 13)
The three dimensions of impact assessed were correlated across the 13 Exemplars. A moderately strong positive correlation was identified between impact on awareness and impact on practice (τ = 0.56) and between impact on policy and impact on practice (τ = 0.44) but not between impact on awareness and policy (τ = 0.06: policy makers are possibly sufficiently aware of the ES concept, such that mainly practitioners and stakeholders other than policy makers achieve greater awareness through ES engagement). Associations between Exemplar characteristics and impact were manifold: studies that addressed a more specific problem were positively associated with a larger impact on policy (τ = 0.45) and with impact on awareness (τ = 0.20). Studies that took place at a smaller scale were positively associated with impact on awareness (τ = 0.35). For impact on policy, the impact increased with scale (τ = 0.23). Studies that scored higher with respect to robustness of the analysis were positively associated with impact on all three dimensions: τ = 0.59, 0.50 and 0.57 for impact on awareness, policy and practice respectively. The number of ES categories addressed in an Exemplar was associated with the impact on awareness (0.45).
The indicator measuring the different aspects of stakeholder involvement was positively associated with impact on awareness and impact on practice (τ = 0.42 and 0.39, respectively). The diversity of approaches used to engage with stakeholders was positively associated with impact on practice (τ = 0.49). The diversity of the stakeholders involved was positively associated with impact on awareness (τ = 0.49). The number of stakeholders involved was positively associated with impact on awareness (τ = 0.23) and impact on practice (τ = 0.37).
Exemplars that addressed the effects of policy instruments directly instead of indirectly had a higher impact in all dimensions: the biserial correlation coefficients were r = − 0.78 for impact on policy, r = − 0.44 for impact on awareness and r = − 0.52 for impact on practice-Exemplars that did not address Fig. 2 Comprehensive graphical overview of associations between Exemplar characteristics and attributional impact dimensions. The upper half of the circle represents the different types of impacts and the lower half, the characteristics of the studies. Only the characteristics which are statistically significant are included here. The width of each connection indicates the strength of the Kendall's correlation between study characteristic and impact. Negative correlations are indicated with dashed outline. Specificness refers to specificity of study. Sc, Instr, Sth, Div, Var and Opp, abbreviations stand for scientific, instruments, stakeholders, diversity, various and opposing, respectively (see Table 4 for corresponding characteristics) the effects of policy instruments directly had on average a lower impact in those categories compared to those that did. Interventional studies were associated with a higher impact on policy compared to observational studies (polyserial correlation coefficient r = 0.54) but a lower impact in practice (polyserial correlation coefficient r = − 0.46).
Despite relatively low correlation coefficients (between 0.2 and 0.5) overall, the relative importance of single factors becomes obvious from Fig. 2. In this figure, the upper half of the circles represents the different types of impacts and the lower half, the characteristics of the studies. The width of each connection indicates the strength of the Kendall's correlation between study characteristic and impact (negative correlations are indicated with dashed outline). Scientific robustness shows the highest positive correlation with all types of impacts: awareness, policy and practice. Additional important factors include the number of ES investigated, the specificity of the study, the inclusion of stakeholders at various study stages, the degree of organisation of the stakeholders, the diversity of stakeholders included and the scale of the study. However, the correlation coefficients are in sum not as high; they vary strongly between the impact of awareness, policy and practice, and they show in parts negative correlation to single types of impacts. One illustrative example is, for instance, scale where larger scale is negatively correlated to impact on practice. Whether or not a study includes stakeholders of opposing views shows least correlation with any of the impacts. Similarly, the diversity of approaches to address stakeholders is not highly correlated with impacts.
ES study characteristics associated with impacts: synthesis from the three databases Table 10 presents a summary of results from the IPBES catalogue, the Lautenbach review and the OPERAs Exemplars analyses. For the OPERAs Exemplars, correlations greater or equal to 0.35 are included (* indicates negative correlation). For the Lautenbach review and the IPBES catalogue, significant characteristics and predictors of studies with recommendations/impacts are included (p < 0.1). Characteristics in italics are common to all databases. While there does not exist a single sweeping intervention or study design affecting consistently all dimensions of attributional impact, our findings highlight four characteristics which appear more frequently (see rows with three Xs, Table 10). These are study robustness, integration of policy instruments into study design, stakeholder involvement and type of stakeholders involved. One could argue that these are the minimum baseline characteristics for impact. Our findings further suggest that to achieve impact along specific dimensions, studies must have a bespoke design: the characteristics associated with impacts on awareness, policy and practice were shown to differ. As a corollary, planning for impact matters. If impact on practice is desired, then a study conducted at smaller scale (e.g. local) which engages with many stakeholders may be more effective than a global modelling exercise testing different policy options and instruments.
The questionnaire can aid in structuring assessments, studies and monitoring programs for impact, by providing various elements for considerations. By highlighting topics such as the analysis of uncertainties, offsite effects or the effects of different policy instruments, it helps researchers track the requirements for, and think early about, desired impacts.

Conclusions
A few lessons can be learned from this study, from which we propose a suite of recommendations.
Lesson-learned 1: There are no clear agreed definitions of impacts in ES science. Such lack of consensus in (and confusion between) definitions of effectiveness, impacts and other measures of a study or intervention's effects contributes to obscuring the potential demonstration of outcomes arising from ES science. Recommendations 1: given the diversity of ES, functions, associated intervention and studies as well as the breadth of potential impacts, impact ought to be defined broadly as the attributional and consequential, positive and negative, long-term effects generated through ES science (and by associated interventions) foreseen or unforeseen, intended or unintended. A recognition of the different categories of impacts is also important. Attributional impacts (impacts on policy, practice, awareness or other direct/indirect drivers) refer to transitory effects which ideally, but not necessarily, lead to consequential impacts (real, on the ground effects on biodiversity, ES, ecosystem functions and wellbeing).
Lesson-learned 2: There is no broad and systematic collection of ES science impact. This leads to a limited body of evidence that ES studies or interventions yield impact. Even the IPBES catalogue does not collate broad or diverse evidence of impact beyond policy, let alone consequential impacts. Recommendations 2: systematic reporting is critically needed for both attributional and consequential impacts. With minimal efforts, the existing compendia (e.g. IPBES) could be expanded to include broader, systemic and comprehensive impact reporting. The impact framework (definition, typology and reporting framework) presented in Fig. 1 offers such a flexible framework, and clarifies the depth with which impact is considered. It considers foreseen and unforeseen effects, but also allows for both prospective and retrospective reporting. In parallel, expected or realised impacts statement should be presented alongside abstracts and keywords, in ES focused peer-reviewed journals. Finally, the use of well grounded reporting frameworks, such as the questionnaire (blueprint protocol) introduced here, can help teams engage more with the realm of impact.
Lesson-learned 3: Across the three databases considered here (IPBES, Lautenbach review and OPERAs Exemplars), a few key study characteristics were found to be more frequently associated with attributional impacts: these are study robustness, integration of policy instruments into study design, stakeholder involvement and type of stakeholders involved. We see these as the minimum baseline for impact. Meanwhile, the 5-year investigation of the OPERAs Exemplars enabled greater in-depth investigation of associations between study characteristics and specific dimensions of attributional impact. While many characteristics were associated with impacts on awareness and practice, the characteristics linked with impact on policy tended to be distinct. Recommendations 3: designing studies for robustness and reliability is fundamental to all dimensions of attributional impactstudy robustness was consistently associated with impact on awareness, policy and practice. Meaningful engagement with stakeholders (see Durham et al. 2014 for guidelines) and careful selection of stakeholder types was also consistently associated with attributional impact. To effect specific dimensions of impacts however, studies must have a bespoke design. Studies conducted at smaller scale (e.g. local) and engaging with many stakeholders were shown to be more effective at generating impact on practice than global modelling studies testing policy options.
In this paper, we have demonstrated that there currently exists no clear consensus on a definition of ES science impacts. We have also shown that limited information is collated on attributional impacts (let alone, consequential impacts). Through robust statistical analyses of three extensive databases of ES studies, we show that certain study characteristics are a key for achieving attributional impact (incl. study robustness and stakeholder engagement).
To break the ecosystem services impact glass ceiling; however, a concerted, global and systematic effort to define, assess and collate evidence of both attributional and consequential impacts is urgently needed.
Funding information This work was supported by the EU's Seventh Framework Programme for Research (FP7) as part of the project OPERAs, Grant Agreement No. 308393.
Open Access This article is distributed under the terms of the Creative Comm ons Attribution 4.0 International License (http:// creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.