Background

Approximately 40% of Europe’s land area (EU-28) is agricultural [1]. These agroecosystems make a crucial contribution to the delivery and maintenance of ecosystem services, and themselves rely on many of these services to sustain food production [2]. Climate regulation and pollination are two regulating ecosystem services with particular policy relevance at this time, having been the subject of global agreements or accords in 2015 and 2016 respectively [3, 4]. Agriculture, and therefore agricultural policy, are central to the maintenance of both these services.

The role of agriculture in global climate regulation was specifically highlighted in the Paris climate agreement at the 21st Conference of Parties to the United Nations Framework Convention on Climate Change (COP21) in 2015 [4]. The importance of agricultural soils to the regulation of the global climate has been emphasized by several studies (e.g. [5,6,7]). The role of soils—whether sequestering or emitting carbon or greenhouse gases—mainly depends upon environmental conditions and soil management. The ‘4 per mille Soils for Food Security and Climate’ initiative, launched in Paris in 2015, aims to increase global soil carbon stocks by 0.4% per year to compensate anthropogenic greenhouse gas emissions. It highlights the particular potential of agricultural soils in mitigating climate change [8].

Pollination is both vitally important to agriculture and threatened by it. Crop pollination is provided by insects, particularly bees, living in crop fields and semi-natural habitats in agricultural landscapes [9]. Pollination is important for yields of three-quarters of the world’s major food crops [10] and has an annual global value of between US$235–577 billion (inflated to 2015 US$) [11]. Pollinators provide numerous other benefits to society beyond agriculture, by sustaining populations of wild plants, producing honey and other bee products and supporting cultural values, for example [11]. Declines in bee diversity have been recorded in highly industrialised regions of the world over the last century [11], although long-term data are only available for a few regions of the world. Agricultural intensification is a major driver of these pollinator declines, as it deteriorates the quantity and quality of habitat for pollinators [11, 12]. There is evidence from around the world that crop pollination services degrade as agriculture intensifies or semi-natural habitat is lost [13,14,15]. High-level political commitments have recently been made to support pollinators in a number of countries, and internationally [16].

Support for ecosystem services under the European Common Agricultural Policy

Both climate regulation and pollination can be seen as ‘public goods’—benefits to wider society—delivered by agricultural landscapes. Pollination also directly benefits farmers by enhancing crop yields. There is a clear mandate for the European Common Agricultural Policy (CAP) to support the delivery of public goods from agriculture [17].

The CAP contains various forms of payment to compensate farmers for providing agricultural ecosystem services [18]. Incentive schemes have until recently been largely voluntary, through ‘agri-environment schemes’ under the rural development (second) pillar of the CAP [19]. The effectiveness of agri-environment schemes at delivering ecosystem service benefits remains poorly understood and was recently highlighted as a research priority [19].

In the current CAP programming period 2014–2020, one-third of the annual direct payments to farmers were made conditional upon a set of “compulsory greening measures”, designed to support climate and environmental policy goals [20]. Three main greening measures were introduced, broadly described as crop diversification, maintenance of permanent pasture and Ecological Focus Areas (EFAs) [20]. The last of these, EFAs, requires farmers with at least 15 hectares of arable land to dedicate 5% of their arable land to ecologically beneficial habitats, landscape features and land uses, as defined by the European legislation. The list of 19 specified elements includes field margins, terraces, hedges, ponds, fallow land, nitrogen fixing crops, and ‘catch crops and green cover’. EFAs are the greening measures primarily designed to “safeguard and improve biodiversity on farms” [20], aligned to Target 3a of the European Union’s Biodiversity Strategy, which is to increase the contribution of agriculture to maintaining and enhancing biodiversity [21]. The crop diversification measure was designed to improve soils, and the permanent grassland measure to maintain the carbon sequestration benefits of grassland. However, all three greening measures are expected to deliver multiple environment and climate benefits.

The overall effectiveness of the compulsory greening measures at delivering environmental benefits across Europe has been repeatedly challenged [22,23,24]. EFAs in particular, have flexibility at Member State level and a strong element of farmer choice. Their implementation has favoured productive, or in-field options such as nitrogen fixing crops, catch crops and green cover and fallow land over habitats and landscape features [23, 25].

Quantifying the environmental benefits of Ecological Focus Areas to inform policy

To facilitate the uptake of EFA choices that deliver higher levels of biodiversity and ecosystem service, the European Commission’s Joint Research Centre fostered development of a software tool called the ‘EFA calculator’ [26, 27]. This calculator generates aggregated positive and negative impact scores for ‘biodiversity’ and ‘ecosystem services’, to reflect the expected relative impacts of implementing a given set of EFA options. It was originally designed to be used at farm-level, but it has been used by the European Commission to analyse the potential impacts of EFA implementation at European scale, using region-level data on the uptake of EFA options [25]. This analysis indicated overall that landscape features and land lying fallow appear the most beneficial EFA types for biodiversity, and there is strong potential to enhance their impact by altering management—for example by sowing flowering plant species on fallow land as opposed to leaving it bare. Regions with a large component of catch crops (> 70% of EFA area) had the lowest potential biodiversity benefits, while the greatest potential benefits to both biodiversity and ecosystem services were achieved by regions with high proportions (> 50% by area) of ‘landscape features’, which are essentially represented by hedges.

The EFA calculator uses a semi-quantitative indicator framework. Its system of scoring, fully described by Tzilikavis et al. [27], uses the following terminology:

Features

20 actual habitat, land use or landscape features representing the 19 EFA options. The 20 features defined by Tzilikavis et al. (Table 1 in 27]) closely match the 19 EFA options defined in European policy [20, 25] but some habitat and land use types, such as woodland and field edges, could be classified under more than one EFA option, while the EFA option ‘Other landscape features’ covers several possible landscape features.

Impacts

Using existing classification schemes [28, 29], the calculator defines 39 taxonomic or functional species groups (for example ‘birds of prey’, ‘pollinating invertebrates’, ‘amphibians’) to represent impacts on biodiversity, and 19 ecosystem service impact categories (for example ‘pollination and seed dispersal’, ‘phosphate runoff’ and ‘pest regulation’). Scores are derived for each impact category expected to be affected by each feature.

Parameters

Attributes associated with a feature which, when changed, alter the impact. For example, ‘soil texture’ or ‘ground cover’.

Parameter classes

Levels describing the condition of a parameter. For example, ‘soil texture’ of the feature ‘fallow land’ can be ‘coarse’, ‘medium’ or ‘fine’ and this changes the impact of fallow land on soil erosion rates.

Parameters are identified, using literature review, for each feature and impact combination, and impact scores are then weighted according to the parameter class. Full lists of features, impacts, parameters and parameter classes for each EFA option are provided by Tzilivakis et al. [27]. The maximum level of aggregation of all the scores leads to four separate scores (separate positive and negative impact scores for biodiversity and ecosystem services, respectively) for a given combination of EFA options, at farm or regional level.

The impact scores and parameter weights used in the EFA calculator were generated based on literature review (> 350 papers, reports and guidance documents) combined with either semi-quantitative or qualitative assignment of scores between − 100 and + 100 for each feature-impact combination. The method of literature review is best described as ‘rapid evidence assessment’, or ‘rapid review’, a structured, stepwise methodology using systematic review techniques but with components of the process simplified or omitted to produce information in a short period of time [27, 30, 31].

The policy need to strengthen farm level tools to assess farm performance

To move the CAP towards a more performance based policy, farm level tools such as the EFA calculator should be developed, underpinned by a solid scientific knowledge base. During 2017 and 2018, the CAP, including its greening measures, are being evaluated by the European Commission in a process of impact assessment, with a view to formulating the policy for the period following 2020. The EFA calculator represents an important tool for this evaluation, since no other methods exist to evaluate the impacts of greening measures across a range of environmental benefits. However, it has two specific shortcomings that require further work, to improve its legitimacy, transparency and credibility.

First, the process of developing the EFA calculator identified major gaps in evidence found using the rapid evidence assessment method. For example, evidence was only included to address impacts on ‘global climate regulation’ for a single EFA feature—woodland, equivalent to two EFA options, ‘trees in groups and field copses’ and ‘afforestation’ [27]. Such evidence gaps may be genuine gaps in knowledge, or result from missing search terms or missed papers during the rapid review process, an evidence synthesis method characterized as having a medium risk of bias and no internationally recognized standards [30, 31]. The climate regulation example is a particularly prominent shortcoming, because climate regulation in agriculture is a high policy priority internationally (see above).

Second, the process of assigning impact scores in the EFA calculator is relatively subjective. Although a few impacts were evaluated using semi-quantitative modelling [27], most of the scores are based on a process of expert judgement. This is described as a general framework, but without a detailed protocol, so the reliability of the weightings and scores is somewhat difficult to assess.

A systematic map has been commissioned by JRC to strengthen the evidence base and methodology underlying the EFA calculator, in order for an improved version to be applied in the forthcoming period, to help define and implement a policy based on reliable evidence. The opportunity to influence reforms of the CAP greening measures, and the EFAs, both at European and Member State level, is a short-lived policy window [32]. At the end of 2018, it will close and not open again for several years. There is therefore considerable urgency for the outcomes of this systematic mapping exercise. This has influenced the protocol, by confining the number of EFA options, ecosystem services outcomes that could be included, and substantially constraining the number of databases that can be searched.

The authors from JRC (Angileri, Paracchini and Terres) selected the combinations of habitat and landscape features and ecosystem service impacts that form the focus of the systematic map. It is a small subset of the combinations available, heavily constrained by the limited time and resources to complete the systematic map in time for the policy need. The selection focussed on thematic areas with knowledge gaps in the EFA calculator current version. Selected combinations are shown in Table 1. Two ecosystem service impacts—climate regulation and pollination—were identified for their high policy relevance. A set of EFA options incorporating woody biomass, but not covered in the EFA calculator for these ecosystems services (hedges, trees in line and agroforestry), were selected as these are expected to contribute to global climate regulation by sequestering carbon. The productive EFA options (cover crops and nitrogen fixing crops) were included because of their high implementation rates (> 70% of the EFA area for 2015) and the expectation that they have relatively low benefits for biodiversity, the primary goal of the EFA greening measure [20, 23, 25]. Fallow land is also very widely implemented (almost 26% of the EFA area; [25]). Pollination impacts were only included for one EFA option, nitrogen fixing crops. This EFA option is likely to be closely scrutinised and challenged in the post-2020 CAP reform process, as it is the most popular with farmers, but considered unlikely to have large biodiversity benefits [23, 33]. It may, however, have benefits for soil carbon sequestration, greenhouse gas emissions or pollinators. Evidence for these benefits has not yet been rigorously synthesized.

Table 1 Ecological Focus Area (EFA) features and ecosystem service impacts selected for systematic mapping
Full size table

A related systematic map is currently underway, looking at impacts of managed vegetated strips on multiple ecosystem services [34]. This is entirely complementary to the present protocol, as it considers a different set of impacts (nutrients, pollutants, socioeconomics, biodiversity, and soil retention; not pollination or climate regulation). Haddaway et al. [34] largely cover different EFA options, focusing on field margins, buffer strips and forest edges, overlapping with this protocol only on hedges and trees in line.

Objective of the systematic map

The main objective of this systematic map is to provide an overview of the available evidence about the impacts of the selected EFA features in agricultural land on pollination and/or climate regulation services (Table 1). The systematic map will describe the volume and nature of the evidence base, enabling the commissioning organisation (JRC) to update and adjust the impact scores in the EFA calculator and identify knowledge gaps with greater rigour.

The map will focus on studies undertaken in Europe and neighbouring countries, including those bordering the Mediterranean Sea. This geographic scope is justified because the EFA features incentivised by the European Union are widely used in farmed landscapes in other countries. Ecological evidence of their ecosystem service impacts from closely located countries in similar biogeographic contexts is highly relevant.

The map will include all agricultural landscapes, even though EFAs apply only on arable land and are currently only required on farms with at least 15 ha of arable land (see Inclusion criteria below). This extension is because there is every chance the EFA policy will change in the next CAP, and be extended to cover all farming systems. We will identify articles that study the parameters and parameter classes included in the EFA Calculator scoring system, but also look for other policy-relevant parameters not currently included. For example, use (or not) of pesticides on nitrogen fixing crops is not included as a parameter in the EFA calculator, but it has recently been accepted as a change to the EFA policy [35].

Primary question

What are the impacts of selected EFA features in agricultural land on pollination and/or climate regulation services?

Secondary question

Which parameters and classes of the EFA feature have an impact on the target ecosystem service delivery?

Population

The agricultural ecosystem in Europe, including ecosystem service stocks and flows.

Intervention/exposure

The selected EFA features and corresponding parameters and parameter classes.

Comparator

For the primary question, the comparator is either land without the EFA feature, or land before it was created or implemented. For the secondary question, the comparator can be different EFA parameters.

Outcome

For the selected ecosystem services, we have already identified the following metrics to represent either the stock, or flow of climate regulation or pollination services, or proxy measures assumed to quantitatively reflect these services (Table 2). Studies reporting any of these metrics will be included. Other metrics may be added iteratively, as they are identified within the relevant literature, and will be coded accordingly.

Table 2 Anticipated outcome metrics
Full size table

Methods

Searches

Bibliographic databases

The following academic citation databases will be searched for studies:

  1. 1.

    Web of Science Core Collections (http://www.wok.mimas.ac.uk/).

  2. 2.

    Science Direct (http://www.sciencedirect.com/).

  3. 3.

    Scopus (http://www.scopus.com/).

  4. 4.

    CAB Direct (https://www.cabdirect.org/).

We will not search the internet databases such as Google Scholar, or specified websites, for grey literature. This is because the European Commission, who requested and funded the project, has expressed a strong preference for the outcomes of this work, including the contents of the systematic map and the derived impact scores for an updated version of the EFA calculator, to be based on peer-reviewed, published scientific literature. We appreciate that this limited search strategy is likely to miss relevant evidence, including some peer-reviewed evidence in Ph.D. theses, books and non-English language literature, for example. The results are also likely to be more biased towards evidence from north-western Europe than would be the case for a more extensive systematic map. The search strategy is streamlined to allow results in good time for the current, time-limited opportunity to influence reforms of the CAP greening measures at European and Member State level. Its limitations are clear to all authors, including those from JRC who commissioned the systematic map, and must be taken into account when interpreting the results.

Search strings

The search strategy has been optimized during a scoping phase, aiming to find an appropriate balance between sensitivity (collating all relevant information) and specificity (the proportion of articles that are relevant) [36]. Search terms were identified based on the terms used in the review process for the EFA calculator’s development [26], and complemented through an explorative search. Relevant pollinator species and nitrogen fixing crop types were selected in line with recent literature reviews [33, 37]. The search terms were combined into search strings using wildcards (* and $) and connectors (AND, OR and NEAR/n or W/n). The ‘*’ wildcard allows to pick up multiple word endings, e.g. pollinat* would find pollination, pollinator, pollinators, etc. The wildcard ‘$’ represents zero or one character and will be used to search for plural forms of search terms, e.g. hedge$ will pick up hedge and hedges. This wildcard will only be used to search the Web of Science Core Collection database, as searches in Science Direct and Scopus automatically retrieve plural forms. Search terms will be combined using the Boolean operators AND (both terms must appear somewhere in the search fields) and OR (at least one term must appear). The proximity operator NEAR/n (Web of Science Core Collection) or W/n (Science Direct and Scopus) will be used to find records where terms joined by this operator appear within a specified number of words (n) of each other. For example, “climate change” NEAR/1 mitigat* will pick up ‘climate change mitigation’, ‘mitigate climate change’, etc.

The comprehensiveness of the tested search strings was assessed using a total of 37 relevant articles (test papers) identified through an explorative database search complemented by snowballing and personal knowledge. Individual search strings were developed for each of the seven EFA feature—ecosystem service combinations, which returned all of these test papers for that combination, from two academic databases: Web of Science and ScienceDirect. The seven final search strings are shown in Table 3. The results of the scoping process through which they were developed, including the test papers, are reported in Additional file 1.

Table 3 Proposed search strings
Full size table

Optimal cut off dates were determined using the ‘Hedges or wooded strips—climate regulation’ combination. Three different cut off dates were tested: 1980 (in line with Tzilivakis et al. [27]), 1995 (justified by the rapid increase in ecosystem service and agri-environment scheme research after that date) and 2005 (publication of the Millennium Ecosystem Assessment). The year 1995 was selected as the most appropriate cut-off date. Before 1995, few papers were returned that were relevant to ecosystem services, which is in line with the findings by Fisher et al. [38] who reported an exponential increase in the number of papers on ecosystem/ecological services after that date.

Article screening and study inclusion criteria

All articles identified through searching will be screened at title and abstract level for relevance using predefined inclusion criteria (detailed below). Articles for which the inclusion criteria cannot be tested at this level (e.g. when the location is not clear from the title and abstract) will be screened at full text level. All screened full texts that are excluded from the systematic map will be listed along with exclusion reasons in the final report. Consistency in the application of the inclusion criteria will be tested by comparing agreement between two reviewers at title, and abstract level screening, using a subset of 140 abstracts, i.e. 20 abstracts for each EFA feature—ecosystem service combination. Disagreements will be discussed and the inclusion criteria adjusted accordingly. A Kappa analysis will be performed to test agreement between reviewers. If the agreement score falls below 0.6, indicating substantial disagreement, a third reviewer will be consulted and a further 140 abstracts screened following discussion of disagreements.

Location and type of landscape

Studies must be based in an agricultural ecosystem (arable, pastoral, mixed farming and perennial cropping systems will all be included) within one of the countries in continental Europe or surrounding the Mediterranean Sea: Albania, Andorra, Armenia, Austria, Azerbaijan, Belarus, Belgium, Bosnia and Herzegovina, Bulgaria, Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Georgia, Germany, Greece, Hungary, Iceland, Ireland, Italy, Kazakhstan, Kosovo, Latvia, Liechtenstein, Lithuania, Luxembourg, the former Yugoslav Republic of Macedonia, Malta, Moldova, Monaco, Montenegro, The Netherlands, Norway, Poland, Portugal, Romania, Russia, San Marino, Serbia, Slovakia, Slovenia, Spain, Sweden, Switzerland, Turkey, Ukraine, United Kingdom, Vatican City, Morocco, Algeria, Tunisia, Libya, Egypt, Israel, Palestine, Lebanon, Syria.

Interventions

Presence of the selected EFA features: hedges or wooded strips, agroforestry, trees in line (including shelterbelts and tree belts NOT orchards), fallow land, catch crops or green cover and nitrogen fixing crops (NOT clover as part of a mixed plant community in permanent grassland). The features will not have to be officially classed as EFAs for the purpose of the study, but must match the characteristics described by the EFA policy [25].

Comparators

In general, studies will only be included if they compare outcomes with at least one of the following comparators: before or without EFA feature establishment; before or without incorporation of EFA land use (e.g. nitrogen fixing crop, catch crop or fallow land) in a crop rotation; alternative levels of moderators (EFA feature parameters) including: tillage treatment (e.g. no-till, minimum tillage, conventional tillage), fertiliser use, crop residue management, crop and tree species and mixture, sowing date, soil type, crop variety. An exception will be made for studies that measure the proportion of nitrogen fixing crops in the diet of potential pollinators. This is a proxy measure of benefit to pollinators that does not require a comparator.

Outcomes

Measured or estimated greenhouse gas emissions including CO2, N2O and CH4; above- or belowground carbon content (%), stock (kg m−2) or sequestration (kg m−2 year−1); modelled pollination service; abundance/diversity of potential pollinators (flower-visiting insects); seed set, fruit set of crop plants (including non-nitrogen fixing crops in the same landscape) or wild flowering plants; proportions of nitrogen fixing crop pollen in the diet of potential pollinators.

Type of study design

Empirical field study including site comparisons, before-and-after trials (see Table 4); predictive modelling study; quantitative meta-analysis. Review papers comprising published data only, without quantitative meta-analysis, will not be included.

Table 4 Proposed study design terms for data coding
Full size table

Language

Only studies published in English will be included.

Study quality assessment

Full critical appraisal will not be part of this systematic map as methods will vary substantially across the included studies. Study design, described using standardized terms (Table 4), will provide some indication of quality.

Data coding strategy

We will extract standardized descriptive data from all studies meeting our inclusion criteria. Data will be stored in a MS-Excel spreadsheet, which will form the systematic map. Data extracted from each study will be coded as follows:

Bibliographic information

Unique reference ID, Reference type, Year of publication, Authors, Title, Journal, Volume, Page numbers, URL or DOI.

Study information

Period of study, Study location name, Country/continent, Study design (from a set of standard terms defined below); EFA feature(s) addressed (Table 1); Ecosystem service addressed (Table 1); comparator—alternative land cover option (primary question) or EFA parameters or parameter classes (secondary question); outcome metric (Table 2).

Table 4 defines terms that will be used to describe study design, largely following those used by the Conservation Evidence project to summarize studies [39, 40].

For comparators (alternative land cover options and parameters) and outcome metrics, a set of standard terms will be developed to define the options available in each database field, as the systematic map progresses. These will follow the terms used in the EFA calculator [27] as far as possible, although new parameters and metrics can be introduced if they are present in the literature.

To test the repeatability of the data coding strategy, two reviewers will independently code a sample of 35 papers, comprising five full texts for each EFA feature—ecosystem service combination. If there is disagreement over inclusion at full text, or different categories are extracted, a third reviewer will code the same papers and all three will discuss the inconsistencies.

Study mapping and presentation

All included studies and their meta-data will be recorded in a MS-Excel database that will be made available with the published systematic map report, as an additional supporting file. The accompanying report will describe the review process, the amount and nature of available evidence, knowledge gaps and recommendations for future research.

Since the objective of this systematic map is to enable the commissioning organisation (JRC) to update and adjust the impact scores in the EFA calculator [27] and identify knowledge gaps with greater transparency and rigour, a final section of the systematic map report will consider and assess the available methods to convert elements of the systematic map into impact and weighting scores. This will include identifying EFA feature-impact combinations that are sufficiently covered by existing quantitative studies to allow full systematic review (including critical appraisal and meta-analysis); identifying EFA feature-impact combinations where the evidence includes quantitative models that could be used to generate scores directly; and describing a trial of the ‘summary and expert assessment of evidence approach’ demonstrated by Dicks et al. [41], in which concise, plain English summaries of each paper (summaries and synopses method; [30, 31]) are reviewed by a multi-expert panel, following a modified Delphi process to generate scores [30].