Introduction

While biodiversity loss is still continuing at a global scale (Tittensor et al. 2014), the need for evidence based decision-making is more and more recognized in the environmental sector (Holmes and Clark 2008; Rands et al. 2010; Perrings et al. 2011; Díaz et al. 2015; Lundquist et al. 2015). To increase the policy impact and ultimately the societal impact of ecological research, a greater emphasis on dialogue and mutual learning between researchers and decision-makers is required. Such dialogue must encompass the complete cycle of knowledge generation through scientific research, policy design and its implementation (which, in turn, involves the generation of practical knowledge and know-how by decision-makers and practitioners). Also, a better framing is needed at the science–policy interfaces (SPIs) to increase transparency, address potential limitations and biases in procedures, and evaluate the progress made in such collaborative undertakings (Hulme et al. 2011; Nesshöver et al. 2013, 2016; Carmen et al. 2015).

Scientific knowledge on biodiversity and ecosystem services is often underused for decision-making, because the functioning of most SPIs is constrained by communication barriers arising from sectorial, educational and cultural differences between its two main counterparts (Spierenburg 2012; Young et al. 2014). On the one hand, scientific knowledge is often too theoretical, hidden by profuse jargon, scattered across multiple disciplinary journals that are typically difficult to access, and inconclusive or providing contrasting conclusions (Nesshöver et al. 2013, 2016). Moreover, scientific evidence is reported in a rapidly growing number of publications that are increasingly difficult to survey and read thoroughly. On the other hand, policy-makers and other stakeholders have to take decisions on a short timescale, and they often base their decision on personal knowledge, professional experience or the opinion of a single expert (Sutherland and Burgman 2015; Nesshöver et al. 2016). Knowledge requirements of policy-makers and stakeholders are often task-oriented, practical, and related to a current problem requiring urgent action. Fulfilling these requirements demands an explicit and proactive effort by scientific experts and other knowledge holders, applying suitable methods in close collaboration with the end users of their knowledge (Young et al. 2014; Sutherland and Burgman 2015; Pullin et al. 2016). A functional SPI must thus ensure dialogue conducive to the provision of relevant and objective knowledge in a format that decision-makers can use (Young et al. 2014; Sutherland and Burgman 2015), as well as adequate support to the generation of such knowledge.

Knowledge synthesis is a way to improve science–policy knowledge exchange, which has been developed in several large scale programmes such as the Millennium Ecosystem Assessment (2005), reports from the Intergovernmental Panel on Climate Change (IPCC), and future reports from the Intergovernmental Science–Policy Platform on Biodiversity and Ecosystem Services (IPBES). At the European scale, a recent effort to foster knowledge synthesis was undertaken by the European Network of Knowledge for Biodiversity and Ecosystem Services (NoK, hereafter also named BiodiversityKnowledge) that developed a knowledge synthesis process of a European SPI for Biodiversity issues aimed at providing a transparent structure to receive and answer requests from policy-makers and stakeholders (KNEU Team 2014; Livoreil et al. 2016; Nesshöver et al. 2016). The knowledge synthesis process explains the roles of the several actors involved and the various steps to be implemented, and it was initially developed around three methodological frameworks: expert consultation, evidence-based approaches, and collaborative adaptive management (Livoreil et al. 2016). These can however be enlarged to other approaches for synthesis, such as those explored by Pullin et al. (2016).

In this paper, we summarize the results of three trial assessments used to test the knowledge synthesis process developed for BiodiversityKnowledge to answer requests on EU environmental policy. The three trial assessments aimed at covering broadly different policy sectors and dealt with: conservation and sustainable management of kelp ecosystems, the biological control of agricultural pests, and the conservation and multifunctional management of floodplain landscapes. Our objectives were (i) to conduct knowledge syntheses on these topics by using the developed process, (ii) to assess the strengths and weaknesses of the process in delivering these knowledge syntheses, and (iii) to build on these results for improving the knowledge synthesis process and establishing a functional network of knowledge with evident added value (cf. Nesshöver et al. 2016; Görg et al. 2016). The trial assessments also provided the basis for an assessment of the performance of BiodiversityKnowledge based on interviews with expert group members, requesters, and other participants of the trial assessments (Carmen et al. 2015).

Methods

The topics of the trial assessments

Three trial assessments (‘marine case’, ‘agricultural case’ and ‘conservation case’ hereafter) were conducted. The topic of the marine case, chosen as an example of horizon scanning, i.e. the identification of emergent issues by scientists or stakeholders (Sutherland and Woodroof 2009), was ‘Current trends in kelp forests in Europe and their effects on biodiversity and the provision of ecosystem services’. It was chosen because kelp-dominated assemblages play fundamental ecological roles in temperate and polar coastal areas around the world (Mann 2000; Steneck et al. 2002), providing habitat, food and shelter for other organisms and supplying many ecosystem services (Duggins et al. 1989; Norderhaug et al. 2005; Reisewitz et al. 2006). For this reason it is concerning that kelps are receding in several regions due to the impact of harvesting and other anthropogenic pressures (Smale et al. 2013; Brodie et al. 2014). While a number of local observations suggest that this may be the case also in Europe, the existing knowledge about the status and trends of European kelp forests and the ecosystems services they provide is limited and fragmented (Araújo et al. 2013; Smale et al. 2013). The goal was to gather and synthesize all the available evidence, identify key issues and knowledge gaps, and derive a strategy for the management, monitoring and scientific study of European kelp bed ecosystems. Contacts were made with DG Environment and DG Mare of the European Commission, who declared their interest on effects of kelp changes on fisheries, which became one main focus of this assessment.

The agricultural case focused on the question: ‘Which types of landscape management are effective at maintaining or increasing natural pest regulation?’ The goal was to identify conservation management practices that enhance the populations of pest-control agents, and potentially enable a decrease of pesticide use in agriculture (Pimentel et al. 1992; Kleijn and Sutherland 2003; Veres et al. 2011). Contacts were made with the French Ministry in charge of Agriculture, Food processing and Forestry, the French Ministry in charge of Ecology, Sustainable Development and Energy, and the Austrian Federal Ministry of Agriculture, Forestry, Environment and Water Management, whose policy-makers provided the topic of the request. A systematic review approach was initiated to assess which of the conservation management practices for pest-control agents were effective and under which conditions.

The topic of the conservation case was ‘Impact of multifunctional floodplain management on biodiversity and ecosystem services’. The trial assessment was requested by the DG Environment of the European Commission, because multifunctionality is a key element of Green Infrastructure (Benedict and MacMahon 2002), a framework currently widely used by the European Commission in relation to climate change, cohesion and biodiversity policies (COM 2009, 2011a, b; EEA 2015). Floodplains of large lowland rivers are suitable landscapes for investigating the impact of management options on multiple ecosystem services and landscape multifunctionality, because the potential for the supply of multiple ecosystem services and the need for sustainable landscape management are particularly high (Tockner and Standford 2002; Nijland and Menke 2005; Scholz et al. 2012; Mueller et al. 2014; Schindler et al. 2014). The goal was to review impacts of floodplain management options on biodiversity and to link them to multifunctionality in terms of ecosystem services.

The knowledge synthesis process

The knowledge synthesis process consists of eleven successive steps, linked by feedback loops, throughout three major stages: preparation, conduct and finalization (Livoreil et al. 2016). It involves several types of actors, such as requesters, working group members, members of a Knowledge Coordination Body (KCB), and evaluators. In the preparation phase, the KCB receives a request with certain budget estimation and a specified timeframe, and develops a dialogue with the requesters to refine the initial question (scoping phase; Pullin et al. 2009; Young et al. 2014). Based on a first estimation of available knowledge and evidence (from the literature and other sources), a scoping team estimates the time and resource allocation required to answer the request. Then a call for tender is launched to select a working group to conduct the knowledge synthesis. In the conducting stage, this working group produces a “protocol document” with a detailed methods section, subjects it to external peer-review and circulates it to the network of knowledge for an open consultation process. Based on the protocol approved by the KCB, the working group then conducts the knowledge synthesis and produces a “draft-report” on its results. In the finalisation stage, the draft report is peer-reviewed and circulated for an open-consultation (like the protocol was) and after modifications and approval by the KCB the final report is delivered to the requester. It answers to the initial request, contains recommendations for policy and management, specifies knowledge gaps and limitations identified during the whole procedure, and recommends further research.

These stages, actors and processes were followed in all three trial assessments, except for two important aspects. First, given that the network of knowledge was in a test stage, the trial assessments did not respond to questions raised a priori by external requesters—as would normally happen once the network of knowledge is officially in place. Instead, the questions were developed ahead of time by members of the BiodiversityKnowledge project consortium interacting with policy-makers to obtain potential requests for the trial assessments (Schindler et al. 2013b). Second, all working group leaders and KCB members were experts from the BiodiversityKnowledge project consortium rather than recruited through an open call for tender.

Methods of knowledge synthesis

A broad variety of methods is available for conducting assessments in the knowledge synthesis process (Pullin et al. 2016). We limited our methodological portfolio to expert consultations, systematic reviews, and collaborative adaptive management so that the three trial assessments could act as replicates and deliver lessons on comparable grounds. Expert consultations can be used for all questions and circumstances, using different levels of involvement (such as feedback loops via review), and are the preferred method when tackling a topic within a short period of time or when published knowledge is not directly available (Sutherland and Burgman 2015; Pullin et al. 2016). Systematic reviews are syntheses of published evidence using an objective, transparent, replicable and updatable methodology that relies on a comprehensive collection of studies, quality assessment (evaluation of biases, validity) and calculation of the strength of evidence taking into account the quality of each study (CEE 2013). Collaborative adaptive management is a framework that focuses on decision-making under uncertainty, through the collaborative design of an iterative process of learning from interventions (“learning by doing”) with strong stakeholder involvement. It typically involves one or several processes of knowledge syntheses, explicitly acknowledges the key role of uncertainty in decision-making, and aims at fostering learning from the monitoring of provisional strategies and their iterative adjustment in the light of new information (Holling 1978; Walters 1986; Lee 1993).

Results

Although following the same framework, the knowledge syntheses carried out in three trial assessments differed according to the methods applied, the requesters involved, the availability and commitment of experts that formed working groups, and the outcomes achieved (Table 1).

Table 1 Comparison of the three trial assessments and the knowledge syntheses conducted within them
Full size table

Results of the three trial assessments

The marine case conducted three different knowledge syntheses, addressing different but complementary questions: (1) a systematic review protocol for the question “What is the impact of kelp forest density and/or area on fisheries” (Araújo et al. 2013) and a literature review addressing the effects of changes in kelp species on fisheries (Bertocci et al. 2015) (Fig. 1a). (2) An expert consultation addressing the current status and trends of kelp forests in Europe, which involved 54 experts. The result was a synthetic assessment and the collaborative production of maps with the distribution and trends of the main kelp species across their distributional range in Europe (Fig. 1b). (3) A collaborative adaptive management workshop focused on the sustainable management of European kelp forest changes. It involved twelve participants (including researchers, policy-makers, practitioners and stakeholders) and resulted in two outputs: a group model identifying the primary and secondary drivers of kelp forest changes, and a policy brief summarizing the key policy recommendations. During the workshop, the participants reviewed the existing evidence concerning the status, trends and ecological functions of kelp forests; developed collaborative models of the main policy and management actions required to achieve a commonly-agreed goal (the sustainable management of kelp forests); identified the main knowledge gaps and uncertainties under which current policy and management regimes must operate; and suggested a number of recommendations for future action. The recommendations included in the policy brief (Supplementary Material 1) addressed the monitoring and conservation of European kelp forests, the evaluation of their socio-economic importance, the build-up of public awareness through citizen science and outreach initiatives, and the development of adaptive management strategies.

Fig. 1
figure 1

Selected results from the marine case a number of published studies included in the systematic review, for five groups of commercially valuable species associated to kelp forests (n = 53). The cumulative percentage exceeds 100 % as some studies involved more than one group; b geographical coverage of data provided by experts on the occurrence and/or population trends of eight European kelp species (Araújo et al. 2016). Grey presence/absence data only. Black data on population trends (right panel)

Full size image

The agricultural case achieved the development, submission and peer-review of a systematic review protocol on the topic ‘Which type of landscape management are effective at maintaining or increasing natural pest regulation in a context of decreased use of pesticides?’, preliminary results on the evidence for effects of flowering strips on biodiversity, predators, pests and crop yield (Fig. 2a), and the publication of a flyer on the agricultural case.Footnote 1 Relevant literature was further transferred to Conservation Evidence at Cambridge University (see Dicks et al. 2016) and integrated in a synopsis of evidence on the same topic (Wright et al. 2013). Further, a workshop was conducted to exchange with stakeholders and allowed to conduct a prioritisation exercise on the different management practices identified in the literature (Fig. 2b).

Fig. 2
figure 2

Selected results from the agricultural case: a structure of the chain of causation in relating flower strips to pest control. No study was found that investigated the entire logical chain from biodiversity enhancement in flower strips to an increase in crop yield (dotted arrow). Many studies dealt with biodiversity of flower strips, relationships between flower strips and predators and, to a lesser extent, relationships between flower strips and pests (solid lines). Few studies dealt with relationships between predators and pests and relationships between predators and agricultural yield (dashed arrows); b Nineteen top interventions (out of 86) as prioritized for interest and/or feasibility by stakeholders during the Paris workshop (Dicks et al. 2016)

Full size image

The conservation case achieved the development of a systematic review protocol for the question ‘What is the impact of floodplain management measures on biodiversity and how does the impact vary according to the level of multifunctionality of the measures?’ (Schindler et al. 2013a) and a systematic map including 70 articles addressing this topic (Fig. 3a; Schindler et al. 2013b). It further developed, in the frame of an expert consultation, a landscape-based multifunctionality index to assess the effects of nine bundles of management interventions on 21 ecosystem services provided by European floodplains (Fig. 3b; Schindler et al. 2014). Finally, a second expert consultation was conducted on multifunctional floodplain management and its impacts on biodiversity and ecosystem services for six European countries (Schindler et al. 2016).

Fig. 3
figure 3

Selected results from the conservation case: a taxa/guilds considered in 70 scientific publications included in the preliminary systematic map on the impact of floodplain management on biodiversity in temperate regions (Schindler et al. 2013b) (left panel); multifunctionality index developed for assessing the effects of nine bundles of floodplain interventions on 21 ecosystem services (Schindler et al. 2014). The index ranges between −1 (all ESS are negatively affected) and +1 (all ESS are positively affected) (right panel)

Full size image

Lessons learned

Lessons learned while conducting the three trial assessments are described for each of the three phases of the knowledge synthesis process (preparation, conduct and finalization; Supplementary Material 2). Issues relevant for more than one phase are only described at the first phase they occurred.

Preparation phase

The challenges identified during the preparation phase included the engagement of experts (particularly high-level experts), the scoping of broad requested questions, the identification of existing or ongoing reviews on similar questions, the engagement of policy-makers on questions addressing emerging issues, the limited reputation of the network of knowledge at its very initial stage, and the need for an early communication strategy (Supplementary Material 2).

One important challenge faced was expert availability and commitment. Despite the considerable efforts made during the trial assessments to reach experts, their involvement was not always as strong as required to receive a diverse and reliable feedback. Our experience revealed that personal contact (e.g. per email) worked better than contacting experts via knowledge hubs. Furthermore, face-to-face meetings (e.g. workshops) were an important tool to enhance expert commitment and to share multiple kinds of knowledge – well beyond its more traditional use to share technical information during specific parts of the assessments. Experts and researchers with high reputation, in particular, showed limited interest in conducting extensive parts of the synthesis work. To address this issue and to increase the cost-effectiveness of the whole process, knowledge synthesis should be conducted by mixed teams involving senior researchers (as peer-reviewers and supervisors), junior researchers (for the most scientific aspects of the review process), skilled librarians (for literature searches) and digesters/briefers—supported by evaluators, communicators and knowledge brokers (cf. Sutherland and Burgman 2015). Such teams would greatly benefit from the availability of clear rules for the conduct of the synthesis (like the guidelines for systematic reviews, CEE 2013, and collaborative adaptive management, Williams and Brown 2012).

A second challenge concerned the effectiveness of the scoping phase. Requests from decision-makers are often broad, open framed, exceedingly complex and subjected to various types of uncertainly. Hence, they rarely match the type of questions that can be adequately answered through a straightforward review of the available scientific knowledge. Expectations for getting a precise answer with low uncertainty for such broad questions can rarely be met. BiodiversityKnowledge proposed two complementary solutions to this challenge: On the one hand, broad policy questions can be broken down to a set of smaller, more manageable questions, prioritized based on both their relevance and the feasibility of getting them answered under the actual conditions of time frame and funding (systematic review and expert consultation; Pullin et al. 2009, 2016). On the other hand, policy questions involving high levels of complexity and uncertainty can be addressed through collaborative adaptive management, an iteration of collaborative knowledge-synthesis techniques fostering the generation of new knowledge through the selection, application and monitoring of policies or management strategies (Méndez et al. 2012; Pullin et al. 2016). The scoping phase is critical for choosing and implementing the most suitable approach. Furthermore, it allows working-group leaders, requesters and contributing experts to iteratively negotiate the scope and scale of assessments, a process of paramount importance in ensuring a scientifically credible and policy relevant output (cf. Carmen et al. 2015). An effective scoping phase also serves the purpose of identifying, as early as possible, the list of possible interventions or actions envisioned by stakeholders (CEE 2013); restructuring and refining the request to ensure that the answer will deal with concrete and feasible solutions; involving requesters in all those key decisions, to avoid false expectations regarding outcomes and uncertainties (Carmen et al. 2015); and identifying running or published reviews on the same or similar topics to avoid duplicating efforts (see e.g. the agricultural case).

A third challenge was the handling of emerging questions raised by researchers, NGOs or the general public (i.e., not by policy-makers). For such questions, the lack of interest and participation by policy-makers might hinder the uptake of results (Carmen et al. 2015). Encouraging the dialogue with requesters and the co-construction of requests, ensuring convincing and targeted outputs and preparing a thorough dissemination plan might help in addressing this challenge.

A cross-cutting challenge was the limited reputation of the network of knowledge in its prototype status at the time of initiating and implementing the trial assessments. This limited reputation caused a barrier to the networks credibility, relevance and legitimacy, and this led to the limited involvement and commitment of experts, requesters and stakeholders. Establishing a solid reputation early-on will be crucial when implementing a network of knowledge in order to ensure expert and requester engagement, and thereby a high quality of the assessments. Care must be taken to continuously increase credibility, relevance and legitimacy of the network and its products. Continuous awareness raising and dissemination activities to optimize the network’s reputation are strongly recommended.

Early communication was of key importance to attract the interest of potential actors in the review process (experts, stakeholders and policy-makers) and the trust and support of the target audience. However, care must be taken to avoid advancing results or viewpoints that might become unsupported by evidence during the subsequent review. We therefore recommend developing an integrated communication strategy from the onset, which includes the thorough explanation of goals and involvement options, and the effective communication of the main results at the end of the process.

Conduction phase

The challenges identified during the conduction phase included the need to deal with different types of knowledge, the heterogeneity of knowledge holders and users, the variability of research protocols and results, the reputation of the network of knowledge, and the profile of peer-reviewers (Supplementary Material 2).

Scientific knowledge was easier to access and synthesize than alternative forms of knowledge, such as practical experience or local knowledge. Modular approaches, whereby scientific knowledge is first assessed for instance by means of systematic reviews, and subsequently combined with practical or local knowledge by means of participatory techniques and/or expert consultations, may provide a solution to this challenge. This approach was applied in two of the trial assessments, through the combination of systematic review and expert consultations (conservation case) and the successive application of expert consultation, systematic review and a collaborative adaptive management workshop (marine case).

The heterogeneity of knowledge holders and users represented an additional challenge, but also an opportunity for achieving an efficient exchange and synthesis of knowledge. To increase the chances of success of the working group, it is important to select experts with a wide array of complementary skills—including multidisciplinary expertise, broad geographical coverage and a transdisciplinary mind-set.

The variability of research protocols, type of data collected, documentation and availability of results made it often challenging to achieve a quantitative summary of the available evidence. In questions requiring primary ecological data, limited reporting may compromise the availability of evidence suitable for knowledge synthesis (Haddaway 2015). For broad questions, a landscape of evidence might be drawn as a first step by classifying available studies, and specifying knowledge gaps and uncertainties. In cases of heterogeneous knowledge, it is particularly relevant to critically appraise the collected studies and to choose appropriate methods for summarizing such knowledge (Pullin et al. 2016).

Finalization phase

The challenges identified exclusively during the finalization phase included the limited possibility of incorporating emerging perspectives and research needs, and the limited contribution of experts following the delivery of the synthesis report (e.g. for its dissemination among policy-makers) (Supplementary Material 2).

In the three trial assessments, the process of knowledge synthesis opened up to new perspectives for research and practice and new research questions and synthesis topics emerged on complementary subsets of the literature. In such cases, following the new threads of evidence could prove particularly fruitful, since a major part of the process was already undertaken. We recommend taking into account the possibility of subsequent syntheses that might be relevant for the requesters. This possibility could be facilitated through the identification of complementary funding streams.

Ensuring the commitment of key experts and stakeholders beyond the assessment period is crucial to ensure the successful uptake of policy documents by EU, national and regional policy-makers. This might be particularly important when such documents are entering the policy cycle, since expert opinion is often requested for clarification and further enquiry. Ensuring adequate funding of the finalization phase as integral part of the assessment period is crucial for this purpose.

Discussion

We tested the proposed knowledge synthesis process of BiodiversityKnowledge with three trial assessments that delivered relevant outputs on conservation and management of kelp ecosystems, biological control of agricultural pests, and conservation and multifunctional management of floodplains. The three trial assessments also delivered lessons learned for the future implementation of the network of knowledge. We found that the knowledge synthesis process works in general for the implementation of assessments. In collaboration with requesters and knowledge holders, a number of relevant assessments were produced (Araújo et al. 2013, 2016; Schindler et al. 2013a, b, 2014, 2016; Bertocci et al. 2015; Dicks et al. 2016). The process is user friendly and flexible, though the methods to be applied are subjected to the constraints of financial resources and time frame (Pullin et al. 2016).

Adequate funding and timing for assessments is a key issue to fully exploit the added values of the network of knowledge. In the three trial assessments, resources played a key role in methodological choices and constrained the capacity for a full implementation of the methodologies that can deliver most robust results but also imply more effort than simpler approaches. Rigorous methodologies such as systematic reviews and collaborative adaptive management are often required, because they ensure robust and reliable knowledge reviews and synthesis that are less susceptible to bias, whereas simpler but less robust methods, notably expert consultation, can be implemented under policy contexts with financial and time constraints (Pullin et al. 2016). European-wide systematic reviews could only be framed and initiated (Araújo et al. 2013; Schindler et al. 2013a, b) with the funding for the trial assessments, but had to be finalized with additional resources afterwards (Bertocci et al. 2015; Araújo et al. 2016) or complemented by expert consultations, that delivered less rigorous results on a shorter time frame (Schindler et al. 2013b, 2014). In the agricultural case, the range of possible interventions and the number of stakeholders was too large and the literature was too heterogeneous for a systematic review given the time, funding and personnel available, and a systematic map (Pullin et al. 2016) would have been a suitable product to be aimed for. Overall, the trial assessment showed that a thorough scoping exercise that carefully tailors the breadth of the question to the resources available to address it is of key importance for a successful knowledge synthesis.

The familiarity of experts and stakeholders with the highly structured and more demanding methodologies (i.e. systematic review, collaborative adaptive management) is also a point requiring careful planning and consideration. For simpler methods, notably expert consultation, we experienced higher level of expert engagement and was easier to ensure voluntary contributions. However, differences in engagement might not only be related to the complexity of a methodology, but also to the role that experts are playing, which is, for instance, essential in expert consultations, but only complementary to the review team in systematic reviews. Adequate support by complementary staff (e.g. librarians, facilitators, knowledge brokers) and effective capacity building is of foremost importance to be able to attract high-level experts, who may otherwise be reluctant to devote extensive time periods to any given knowledge review. Adaptive management, in particular, was able to perform well when decisions had to be taken in the face of limited or complex knowledge and uncertainty (cf. Méndez et al. 2012), but experts and stakeholders often lack familiarity with this methodology. A further challenge is this context is the reluctance of policy-makers to explicitly design long-term adaptive management frameworks for their interventions.

The main lessons learned on the functionality of the knowledge synthesis process presented here largely match the conclusions obtained by the independent evaluation by Carmen et al. (2015), structured along four major topics: inclusiveness, effective communication, policy usability and capacity building. This evaluation highlights the acknowledgement and enthusiasm of the trial assessments’ participants for the objectives of integrating different knowledge types, meeting the needs of decision-makers and supporting the build-up of a ‘biodiversity community’. Also Carmen et al. (2015) conclude that understanding the motivation of participants holds the key for the future success of the network of knowledge, and point out to the commitment of policy communities and the funding procedure as issues not entirely solved. Stakeholder engagement and capacity development were also recently identified as major challenges for IPBES (Lundquist et al. 2015).

Added value of BiodiversityKnowledge

According to Nesshöver et al. (2016), BiodiversityKnowledge offers three types of added value: First, it provides a one-entry point for requests from policy that cannot easily be tackled via existing pathways and needs direct input from science. Second, it enables the engagement of knowledge holders in synthesis activities and provides a flexible approach for accessing knowledge at appropriate scales (e.g. on the Member State level via local networks and institutions). Third, it allows for the use of a broad range of approaches that go beyond current standard of writing and peer-review. To be successful in reaching these goals, however, high standards of transparency and independence will be required, as they represent the basis for its credibility, relevance and legitimacy (Nesshöver et al. 2016). Our experiences in the trial assessments confirm the second and the third types of added value. In the conservation case, for example, a team of approximately 20 European experts declared their enthusiasm for the possibility of collaborating on a topic of their expertise that integrated their knowledge at different scales (Schindler et al. 2014, 2016). In the marine case, and to a lesser extent also in the conservation case, a multitude of approaches (systematic reviews, expert consultations and collaborative adaptive management) were successfully applied and combined.

Our trial assessments clearly demonstrated the need for diverse expert groups and flexible approaches, necessary for dealing successfully with requests at different scales, with varying levels of complexity, and often subjected to considerable constraints in time and resources (Pullin et al. 2016). A broad coverage of knowledge and expertise is highly relevant, because the evaluation and improvement of EU’s environmental policy require expertise in many different disciplines, spanning a diverse geographic and socio-economic range, and paying attention to the particularities of the political and environmental history that shaped them (Tryjanowski et al. 2011; Pe’er et al. 2014). According to Sutherland and Burgman (2015), expert consultations should rely on groups of carefully chosen experts with diverse composition and age structure, including individuals capable to integrate information from diverse sources and disciplines. Also peer-reviewers should be a mixture of experts on the topic, on the methodologies used, and they should further include communication experts, to evaluate to which extent the draft protocols and reports are understandable for the requester and the target audience (e.g. the policy-maker community, a specific group of stakeholders, the general public). Training or capacity building might be required, although expertise in European countries is rather broad and well specified. Once the network of knowledge is established, it should therefore be possible to find dedicated experts for most topics in each European country.

We cannot directly confirm the added value of a one-entry point for requests, as it did not exist yet when conducting the trial assessments. However, we can confirm, based on positive and miscellaneous experiences from the three trial assessments, that it is highly beneficial to have stable entry points (ideally, a single contact person) representing the requester during the dialogue that shapes each specific request (i.e., the scoping phase; Young et al. 2014).

Strengths and limitations of the trial assessments

In the trial assessments presented here, we could investigate the performance of BiodiversityKnowledge and its knowledge synthesis process under a diverse set of conditions, sectors, requesters, expert teams and methods. Their implementation reflected also different priorities regarding the focus of the assessment: the marine and the conservation case focussed more on running through all phases of the knowledge synthesis process, allowing for a consistent evaluation of all steps and methodologies (Carmen et al. 2015), whereas the agricultural case had a stronger focus on critical reflections on the procedures for instance by dedicated workshops on this topic.

We were not able to test all the elements of the network of knowledge in the trial assessments. Besides the financial constraints and the limited reputation of the network of knowledge as prototype mentioned in previous sections, our assessments were constrained by the unrealistically small size of the network of knowledge staff. For instance, working-group leaders had to play a dual role as KCB members and reducing potential for disagreements between these groups of actors. These constraints are however likely to persist in the initial stages of the forthcoming network of knowledge (existing knowledge hubs, such as IPBES, are also known to face similar problems; cf. Lundquist et al. 2015) and will have to be addressed by its future management.

Conclusions and recommendations

We conclude that the network of knowledge has potential for a good performance in delivering assessments. The knowledge synthesis process is flexible according to required resources and applied methods and adequate scoping during the preparation phase is of paramount importance to maximize the scientific credibility and policy relevance of the output. Main challenges were to reach engagement of relevant stakeholders and a broad array of experts with diverse and complementary skills. High visibility and reputation of the network of knowledge, supported by an incentive system for participation, will be crucial to ensure such engagement. Based on our insights from the three trial assessments, we further recommend (i) investing in the reputation, and the clarity and transparency of the procedures of the network of knowledge to sufficiently engage actors and to build trust among them, (ii) developing guidelines to fully benefit from the combination of approaches in dealing with the heterogeneity of the available knowledge, and (iii) building capacity for the effective consideration of non-academic forms of knowledge and more demanding methods such as systematic reviews and collaborative adaptive management.