Background

Agriculture in Europe has become progressively more intensive over the last 50 years, resulting in increased usage of fertilizers and agrochemicals, changes to cropping practices, land drainage, and increased stocking rates [1]. The risk of soil erosion and water run-off have increased due to soil compaction and reduced organic matter content resulting from intensive agricultural practices. Application of nutrients in excess of plant needs, coupled with run-off from agricultural land has contributed to a decline in water quality [2].

Between 2004 and 2007, nitrate levels across Europe exceeded European water quality standards (50 mg/litre) in 15% of groundwater monitoring stations and 3% of surface stations. Particularly high levels of nitrate are found in surface waters of England, Belgium (Flanders), Netherlands, France (Brittany), Estonia, Northern Italy, North East Spain and Slovakia [3]. A recent study of 4 European river basins (Elbe in Czech Republic/Germany, Danube in 10 countries, Schelde in Belgium and Llobregat in Spain) has shown that 44 out of 500 chemicals tested for in river water were at levels that could have effects on organisms, and 75% of those chemicals were pesticides [4]. Furthermore, it is estimated that 200 million cubic metres per year of sediment are dredged from European rivers [5]. In the United Kingdom (UK), for example, approximately 28% of phosphates, 70% of nitrates and 76% of sediments in rivers are derived from agricultural activities [6, 7]. In addition, UK catchments dominated by agricultural land use have increased levels of bacterial pathogen counts [8].

A decline in water quality has increased water cleaning costs, reduced reservoir capacities and negatively impacted on wildlife and flood defences [9]. Changes in weather patterns may result in further declines in water quality. Climate change scenarios suggest that the UK will experience wetter winters, and warmer, drier summers. Increased extreme weather events may increase the likelihood of heavy rains washing soil and pollutants into river systems and drier summers will concentrate levels of pollutants in rivers [10].

European member states have a policy commitment to tackle water pollution through a number of directives namely the Water Framework Directive (WFD), the Nitrate Directive, the Ground Water Directive and the Bathing Water Directive. In the UK, Nitrate Vulnerable Zones (NVZs) are used to implement some of this policy nationally [10]. During the last 10 years the UK Department for Environment Food and Rural Affairs (Defra) and the Environment Agency (England and Wales) have funded 200 catchment projects at a cost of around 70 million pounds, of which Defra funded 178 [11]. Much of this funding was spent on studies that assessed the efficacy of mitigation measures in delivering an improved water environment [12].

Objective of the Review

In order to inform future decision-making, a need was identified to evaluate the evidence for the effectiveness of five on-farm mitigation measures that may protect or improve environmental water quality [13]. Those measures are slurry storage, cover crops/catch crops, woodland creation, break-up of compacted layers/controlled trafficking and buffer strips. Slurry storage can reduce pollution incidents caused by spills and leaks, and enables slurry to be kept for spreading when crops are best able to uptake nutrients [14]. Fast growing crops grown between the planting of the main crop (cover crops or catch crops) are used to protect the soil against erosion, minimize the risk of runoff, and ensure that nutrients stay in the root zone [15–17]. Woodlands can improve soil structure, reduce water runoff and aid infiltration of water into soils which reduces the risk of pollutants entering into water sources [18, 19]. Confining farm machinery to certain areas of a field to reduce compaction (controlled trafficking) and breaking up compacted soil layers may reduce soil erosion and water run-off [20]. Buffer strips composed of grass and/or trees can improve water quality by physically trapping sediments and associated pollutants and immobilizing nutrients through plant uptake or microbial degradation [21, 22].

This systematic review will assess the effectiveness of slurry storage, cover/catch crops, woodland creation, controlled trafficking/break-up of compacted layers and buffer strips, as on farm mitigation measures, for delivering an improved water environment. The outputs will consist of a searchable systematic map database for all the named interventions, quantitative analysis of each intervention (depending on the availability of suitable data), and a report summarising the findings.

An analysis of the effectiveness of water pollution mitigation measures should help enable decision makers and delivery agencies to better facilitate catchment planning as required under the European Water Framework Directive [23].

Primary question

This study aims to address the following question:

How effective are the following five on-farm mitigation measures in delivering an improved water environment:

Slurry storage

Cover crops/catch crops

Woodland creation

Break-up compacted layers (arable and grassland) and controlled trafficking (grassland)

Buffer strips?

Primary outcomes measured will be:

Nitrogen (drinking water impact and eutrophication)

Phosphorus

Bacteria in bathing and shellfish waters

Pesticides

Sediment loading and colour (in raw water quality abstracted)

Sediment loading and impact on fish and plants

Methods

The methods used in the development of the systematic map and subsequent systematic review analyses will be adapted from the Collaboration for Environmental Evidence Systematic Review Guidelines [24] and from an existing systematic map report [25].

Searches

A comprehensive search will be undertaken using multiple information sources to capture an un-biased sample of literature. The search strategy has been developed to identify both published and grey literature.

An initial scoping search was performed to validate the methodology. Search terms were tested for specificity and sensitivity using the online database Web of Knowledge, and used to indicate the volume of relevant literature. In addition, a few search terms were tested on CAB abstracts and Science Direct. The search terms, number of articles found and general quality of the search results were recorded in a spread sheet (Microsoft Excel) [see Additional file 1]. The results of the scoping search were used to inform the final search strategy.

Table 1 lists the search terms which will be utilised for each of the database and web searches.

Table 1 Keywords and qualifiers to be used in literature search
Full size table

A wildcard (*) will be used where accepted by a database/search engine to pick up multiple word endings. For example pollut* would pick up pollutant, pollution. A keyword may be made more restrictive by the addition of a qualifier e.g. (slurr* stor* AND water qualit*), (slurr* stor* AND water pollut*). The combination of qualifiers and keywords will vary for each intervention, based on the results of the scoping search. The exact keyword and qualifier combinations to be used are listed in the additional file [Additional file 1].

The following online sources will be searched to identify relevant literature:

Electronic databases:

ISI Web of Knowledge involving the following products: ISI Web of Science; ISI Proceedings

Science Direct

Wiley Online Library

Ingenta Connect

Index to Theses Online

CAB Abstracts

Agricola

Copac

Directory of Open Access Journals

Organisational websites:

Department of Environment Food and Rural Affairs online database (UK)

Environment Agency (UK)

Natural Environment Research Council Open Research Archive (UK)

Forestry Commission/Forestry Research (UK)

The Woodland Trust (UK)

Centre for Ecology and Hydrology (UK)

Natural England

Countryside Council for Wales

Scottish Natural heritage

Scottish Environment Agency

Northern Ireland Environment Agency

European Environment Agency

EU Water Frame Directive

European Commission Joint Research Centre

Controlled traffic farming (European site)

Finnish Environment Agency

Ministry of Agriculture and Forestry (Finland)

Swedish Environment Agency

Danish Environment Agency

Ministry of Food, Agriculture and Fisheries (Denmark)

Government Norway Portal

Flemish Environment Agency

Agriculture and Agri-Food Canada

Environment Canada

US Department of Agriculture

US Environment Protection Agency

Agency of the Environment and Energy (France)

Federal Environment Agency (Germany)

Federal Ministry of Food, Agriculture and Consumer Protection (Germany)

Netherlands Environmental Assessment Agency

Department for the Environment, Transport, Energy and Communication (Switzerland)

Federal Office for Agriculture (Switzerland)

Environmental Protection Authority (New Zealand)

Ministry of Agriculture and Fisheries (New Zealand)

Food and Agriculture Organization of the United Nations

Ecologic Institute (European)

EU Cost (European Cooperation in Science and Technology)

In addition, web searches will be performed using the search engines: http://www.Scirus and http://scholar.google.com. The first 50 hits (.doc .txt.xls and .pdf documents where this can be separated) from each data source will be examined for appropriate data. No further links from the captured website will be followed unless to a document/pdf file. Other specific/specialised databases will be searched where identified or recommended by experts within the field. Database and repository searches will be conducted in the English language. Therefore any European Environment Agency or Agricultural Department website which is not searchable in English will be excluded. The potential language bias associated with this strategy was discussed with funders and stakeholders at an initial inception meeting, and was considered acceptable for this review.

Bibliographies of articles viewed at full text will be searched for relevant articles missed by previous searches. For example, Mayar [22] lists over 80 studies that examined the effect of buffer strips on nitrate levels in water. Recognised experts, practitioners and authors will be contacted for further recommendations and for provision of relevant unpublished material or missing data. For example, Corell [26] keeps an annotated and indexed library on the riparian web page, http://www.unl.edu/nac/riparianbibliography.htm, which contains over 890 references relating to “Vegetated Stream Riparian Zones: Their Effects on Stream Nutrients, Sediments, and Toxic Substances”.

The results of each search term on each database will be imported into a separate EndNote X2™ library file. All the database libraries will be incorporated into one library, recording the number of references captured. Using the automatic function in the EndNote X2™ software any duplicates will be removed.

A record of each search will be made to enable a re-run of the search if needed. The following data will be recorded:

Date search conducted

Database name

Search term

Number of hits

Date limits set on records to search (e.g. 1999–2006)

Notes

Study inclusion criteria

All retrieved studies will be assessed for relevance using inclusion criteria developed in collaboration with funders, stakeholders and with subject experts as follows:

Relevant subject(s)

Studies that investigate some aspect of water quality improvement by one of the on farm mitigation measures will be considered for inclusion into the systematic map, irrespective of scale. Stakeholders agreed that the study should focus on temperate countries with similar farming systems to the UK. Those countries will be: UK, Ireland, France, Belgium, Switzerland, Germany, Holland, Luxembourg, Liechtenstein, Denmark, Sweden, Norway, Finland, Austria, Slovakia, Poland, Hungary, Czech Republic, Romania, Lithuania, Latvia, Estonia, Belarus, Ukraine, northern states of the USA, Canada and New Zealand.

Language

Studies published in English.

Date

No date restrictions will be applied.

Types of intervention (mitigation measure)

The following on-farm interventions that aim to improve water quality will be included: slurry storage, cover crops/catch crops, woodland creation, break-up compacted layers, controlled trafficking and buffer strips.

Types of comparator are likely to include

Woodland compared with other farmland uses; farmland with shelterbelt compared to farm land with no shelterbelt; buffer strip compared to no buffer strip; catch or cover crop compared to fallow; loosening of compacted soil compared to no loosening of compacted soil; no wheel traffic compared to wheel traffic (may be measured in pressure); farming area with high capacity slurry storage compared to farming area with slurry storage of low capacity. Studies that compare or observe effects before and after the implementation of interventions will also be included.

Types of outcome

Differences in water quality measured as change in levels of nitrate, phosphorous, bacterial counts, pesticide and sediments will be considered. If cost information is available it will be noted.

Types of study

Any experimental or correlative research study that collects primary data to investigate the effectiveness of one of the named on-farm mitigation measures for delivering an improved water environment will be considered.

Article Screening

The inclusion criteria will be applied by one reviewer to all potential articles at the title and abstract level. Where there is insufficient information to make an informed decision regarding a studies inclusion, then relevance to the next stage of the review process (full text assessment) will be assumed. A second reviewer will examine a random subset of at least 25% of the reference list (up to a maximum of 200 references) to assess repeatability of the selection criteria. Kappa analysis will be performed, with a rating of ‘substantial’ (0.6 or above) being required to pass the assessment. If the Kappa value is low, the reference list will be reassessed against adjusted inclusion and exclusion criteria. For each stage of the screening process the number of references obtained and excluded will be recorded.

Potential effect modifiers and reasons for heterogeneity

The following list of potential effect modifiers has been compiled following discussion with subject experts, funders and stakeholders:

Country of origin

Farm system (e.g. poultry, pig, dairy, grassland, arable, forestry, divided conifer or broadleaved)

Climate (e.g. annual average rainfall values)

Soil properties (e.g. free or poor draining)

Nutrient source (soil, manure/excreta and fertiliser)

Pollutant pathway (Point, diffuse, subsurface, surface flow)

Pollutant soluble or insoluble

Quantity of nutrient addition

Length of time farmed under same regime.

Previous farming regimes on land.

Buffer/Shelterbelt width

Vegetation Composition (grass, mixed grass/woodland/shrub or woodland )

Woodland management (native woodland, short rotation, coppice, agroforestry, energy fuels)

Crop Type

Time of planting

Slurry storage design

Scale of study (catchment, plot)

Water quality measurement (plot, water source)

Distance of mitigation measure from water source

Landscape characteristics e.g. slope (angle/length)

Percentage of land covered by mitigation measure

Data extraction strategy

Systematic map database

Studies that pass the inclusion criteria will be imported into a database. Each article will be coded and categorised using a combination of generic (e.g. country/s of study, publication date, length of study etc.) and topic specific (e.g. percentage nitrate) keywords. Data regarding the study characteristics, quality of design and results will be recorded. A notes section will identify any interesting results such as synergistic effects (e.g. pollution swapping), but will not be included in further analysis as those effects have not been searched for specifically. Where there is more than one article found for a study, each article will be recorded and cross referenced in the systematic map database.

Expert advice together with examination of relevant agricultural ontologies such as the online thesaurus of agricultural terms provided by The Food and Agriculture Organization of the United Nations (http://aims.fao.org/website/AGROVOC-Thesaurus/sub) will be used to assess the suitability of coding terms.

The database (systematic map) will describe the extent of the research in the field and identify knowledge gaps. It will be searchable by topic and can be arranged according to topic areas, publication date, intervention type, pollutant type, country of study etc. Simple numerical accounts of the frequencies in each category can be obtained from the systematic map. Pivot tables can be generated in order to identify trends in the research.

Data will be extracted by one reviewer, and a random subset of at least 25% of the selected studies will be checked by another reviewer to verify repeatability and accuracy. Where information regarding the reasons for heterogeneity is presented in the studies, it will be recorded e.g. climate zone, country etc. Where necessary and feasible, authors will be contacted for missing/suitable data.

Subject experts will review the completed map to ensure that all relevant categories have been defined.

Study quality assessment

Each study included in the systematic map will be categorised according to a hierarchy of evidence adapted from systematic review guidelines used in medicine and public health [27] and conservation [28]. For example, a randomised control trial would be weighted higher than a site comparison study. The hierarchy of evidence will include factors that are both generic to environmental evidence, and specific to water pollution. Categorisation will be based on a ranking system according to the quality of evidence. A generic list, adapted for environmental conservation by Pullin and Knight [29], see Table 2, will be modified and combined with topic-specific quality measures. Topic specific quality measures may include an assessment of the sampling methodology used (e.g. number of samples taken, frequency of sampling, period of sampling, quality of measure, standards adhered to etc). Coding for these methodological factors will be applied during the creation of the systematic map database, and will be used to decide the final categorisation for each study.

Table 2 Generic hierarchy of quality of evidence based on the type of research undertaken for environmental evidence
Full size table

The study quality of all accepted full text articles will be categorised by one reviewer during the coding phase. A second reviewer will examine a random subset of at least 25% of the selected studies to assess repeatability of study quality. Disagreement regarding study quality will be resolved by consensus. No studies will be excluded on the basis of study quality, but will be categorised accordingly.

Data synthesis and presentation

Methods of data synthesis will depend on the type of data presented in the accepted studies. As a minimum, all studies accepted for inclusion will be summarised within the systematic map database. Summary tables of study characteristics, study quality and results will be presented, accompanied by a narrative synthesis.

Where quantitative measures of effectiveness (e.g. percentage reduction in pollutant as opposed to presence/absence data) are present in studies, interventions will be scored for their effectiveness for a particular pollutant. Interventions with a greater impact will score more highly.

For each intervention, the measures of effectiveness scores, will be combined with the quality of evidence categorisations to provide an indication the level of effectiveness and the level of knowledge for each intervention, i.e. an intervention that consistently provides a small nitrate reduction across high quality studies may be categorized as slightly effective for N, with good quality of evidence. An intervention that has been shown to provide a very large reduction in N, but appears low in the hierarchy of evidence may be categorized as having potential effectiveness, but poor quality of evidence, with a recommendation for further research.

In addition to the basic quality and state of knowledge scores, quantitative analysis will be undertaken on any data suitable for formal statistical analysis. Where possible, meta-analysis for each of the interventions will be carried out with reasons for heterogeneity assessed by meta-regression (univariate or multivariate). If meta-analysis is not possible, then other appropriate statistical techniques may be performed.

Recommendations will be made for policy decision making and for future research based on the findings of both the scoring exercises and the meta-analysis. Subject experts will help interpret meta-analysis results and provide input to final recommendations. The systematic map will enable decision makers to search for specific details of interventions and their impacts in differing contexts.