Background

Problem statement

Spring frosts are among the most important causes of yield loss in fruit orchards in temperate regions [1]. Frost damage occurs when the ambient temperature drops below a crop and development stage-specific critical threshold [2]. A single frost night on 19th April 2017 led to apple yield losses of up to 78% in Europe compared to the preceding seven years (Fig. 1) [3].

Fig. 1
figure 1

Relative apple and pear yields in European countries for 2017 in comparison to the mean yield of 2009–2016, as reported by the World Apple and Pear Association [3]

Full size image

Since the 1950s several fruit research stations across Europe report an earlier start of flowering [4] which could potentially lead to higher probabilities for frosts during the flowering period. However, in a warming climate, also the last frost day is occurring earlier in the year and regional studies arrive at contrasting conclusions over the evolution of the frost hazard during flowering. While some studies report on unchanged or increased frequencies [5,6,7,8,9,10,11,12] for certain regions, decreased risks were reported for other regions or by other studies [8,9,10,11,12,13,14,15]. Although studies differ substantially in the employed phenological models, climate models and assessment methodologies, most of them confirm that frost during flowering will remain a major risk factor for fruit production in warming temperate regions.

Research and interests

Similar reviews were conducted in the past, whereby the most prominent reference is a handbook for frost protection, published by the FAO and edited by Snyder and de Melo-Abreu [1]. With the majority of the referenced studies dating prior to 1995, an updated and systematic review seems needed to summarise new research findings and identify gaps. Specialised reviews have recently been published on protection against frost by means of wind turbines [16] and on innovative sensor networks for early frost detection [17]. Another recent and comprehensive summary on frost prevention techniques was published by ‘Les instituts techniques agricoles’ [18], which is geared towards practitioners and lacks academic referencing. In 2018 and 2019 the EIP-AGRI Focus Group collected knowledge of stakeholders and experts from across Europe on fruit protection against frost [19]. Besides a characterisation of strengths and weaknesses of common passive and active interventions, a list of research needs was compiled. First and foremost it mentioned the need for “Studying and comparing the effectiveness of methods under different conditions” (p. 35) followed by “More biology to the models” and “Establishing a database on potential yields for different species/varieties and critical temperatures on species/variety level”.

The interests of stakeholders in this topic converge in a better assessment and understanding of the effectiveness of techniques. The targeted end-users of the review are the fruit producers and their advisors. Other major stakeholders are insurance businesses and governmental agricultural services at the regional, national, and international level as well as academic, national or regional fruit research centres. The Belgian ‘Proefcentrum fruitteelt vzw’, 3800 Sint-Truiden, Kerkom, (PCFruit vzw), a major experimental and applied research station for fruit production, was directly involved in the design of the present review protocol, regarding the selection of databases, search strings, population definition, data extraction and validity assessment criteria. An ultimate group of stakeholders are the consumers of fresh fruit who prefer fruits with low prices and high quality, implying no damage suffered from frost or frost damage remediation treatments.

Objective of the review

Farmers can choose between or combine different kinds of frost risk management strategies: financial (insurances, income diversification) or physical (windbreaks, foliar applications, sprinklers, wind machines, …). This review will focus on the latter kind, provided, that they affect either ambient temperatures or crop development.

This review seeks to answer two research questions. The primary one is “How do the performances of spring frost damage reduction strategies in temperate fruit orchards compare?” while the second is “'How do environmental conditions affect the effectiveness of frost damage reduction strategies in temperate fruit orchards?”.

Thereby the main objective is to compare the effectiveness of the different measures that target at least partially the reduction of yield losses due to spring frost. The identified environmental preconditions shall be used for a classification of the techniques as a basis for informed decision making in the fruit sector. Only tested interventions described in the published literature are considered, thereby excluding patents.

The review will aim to produce a comprehensive inventory of the available evidence of frost protection measures for commercially important temperate fruit tree orchards and vineyards. The studied fruits were selected from a list of the major temperate fruits [20] making use of expert knowledge (Table 1). The review will include all geographical locations that are under a temperate climate according to the Köppen-Geiger climate classification system [21], corresponding to the main climate type ‘C’, see Fig. 2.

Table 1 Elements of the systematic review question
Full size table
Fig. 2
figure 2

Fruit production by botanical fruit group and region, based on FAOSTAT production data for 2019. Yellow shades are Temperate zones after Köppen—Geiger (type 'C'). Regional grouping follows the definitions and standards used in FAOSTAT [22]. Fruit groupings were defined based on both botanical and geographical criteria and listed in Additional file 1

Full size image

We will limit the review to preventing frost damage during early development stages (after bud break in spring) and not during the harvest period (autumn) nor storage. We consider damages during dormancy as well as the temporal evolution or spatial distribution of frost events and associated losses beyond the scope of this review.

Methods

The reporting of this protocol follows the RepOrting standards for Systematic Review Syntheses (ROSES) for systematic review protocols [23]. The completed form can be found in the Additional file 2.

Searches

Bibliographic databases

Several kinds of sources will be drawn upon for this review: academic literature from peer-reviewed journals as well as ‘practitioners’, ‘specialist’ or ‘grey’ literature comprising theses, institutional and governmental reports, which will be identified via dedicated search engines. The selection of relevant databases and journals was based on consultations with librarians of the University of Leuven and fruit research experts, and on the recommendations of the Collaboration for Environmental Evidence (CEE) [24].

Peer-reviewed articles will be searched for based on Title, Abstract and Keywords (Author based and generated) using the advanced search facility of the databases “Scopus” and the following collections in “Web of Science” (WoS): Core collection and regional collections covering publications from Korea (KJD), Russia (RSCI), China (CSCD) and Latin America (SCIELO), as detailed in Table 2, where the temporal coverage of every index is given. No period, document type or language will be excluded at this stage.

Table 2 Overview of libraries to be queried with the temporal coverage and number of hits for the defined search strings
Full size table

Libraries and databases specialised in agriculture including Agricola (United States Department of Agriculture), CAB Abstracts (Centre for Agriculture and Bioscience International), FAO Agris (Food and Agriculture Organisation), Groenekennis (University of Wageningen) will be searched for with time and index settings as detailed in Table 2.

For an article to be returned, at least one term referring to the review population (common name, Latin name or type of fruit) is needed and either a combination of highly specific terms like “frost” and a term of context or action—or a predefined association including generic terms like “cold” or “low temperature”. A successful combination could thus be “apple” AND “protection” AND “frost”. The search string was initially developed for the Web of Science (Table 3) and was then adjusted to the specific syntax of the other databases/search engines (Additional file 3). The Groene Kennis database includes (Dutch) classification tags, although only until 2015, including “fruitteelt” (Fruit production) and “beschadigingen door vorst” (frost damage), which will be searched for separately. By default, the search widget of the Groene Kennis database searches full text, leading to significant noise. With the URLs given in Additional file 3, only the Abstract, Title and Keywords are considered (Rob Genderen, Wageningen University, personal communication on 9/2/2021). Also the CAB Abstracts are searched using the descriptors "frost injury" and "frost protection" while the search in AGRICOLA is amended by searching by the more generic Code 4520 for “Miscellaneous plant diseases, injuries and control”.

Table 3 Complete search strings by database and language
Full size table

Since the Groene Kennis and FAO Agris search results cannot be exported to a .RIS file at once, the bibliometric data will be scraped and written into .RIS files. Entries from FAO Agris will be filtered by the eligible languages at this step.

Estimations of the comprehensiveness of the searches

It became evident during preliminary scoping exercises that an important share of studies was published before 1990. This is confirmed by a recent review on wind-based protection against frost damage [16]. Therefore, no date filters will be applied. The earliest records listed in the employed databases range from 1903 (Agricola) to 1955 (Web of Science databases) (Table 2).

Prior to the protocol definition, 149 references to articles dealing with frost protection at large were extracted from past reviews [1, 16,17,18]. Based on the title only, around two thirds potentially satisfy the eligibility criteria for the review (listed in Table 5). They served as a reference for the formulation of the search strings. A substantial part was retrieved through the Web of Science and the remainder -mostly grey literature- was traced back to their sources or libraries. These were added to the lists of additional databases and websites to be searched (Table 4).

Table 4 Selected websites for specialist search
Full size table

Automated alerts for the defined searches will be activated in Scopus and Web of Science, in order to include papers published during the review period. The other databases will be queried again, including a filter for the publication year 2021 if possible, at the end of the data extraction phase and before the analysis phase.

In the international databases, searches are exclusively conducted in English, since we found that searching with German or French translations of the search strings did not have an added value, due to abstracts translated in English. Publications with English abstracts and full texts in Dutch, German, French and Spanish are thus considered as well. Including five languages in the review process reduces the language bias and the risk of missing out from key literature [25]. While the most prominent publications stem from northern America and Europe, within the available resources we will include publications from fruit growing regions in temperate Asia and Latin America [22], considering their importance for fruit production (Fig. 2).

Specialist searches

Google Scholar and Baidu Scholar are queried in English (see Table 1 in the Additional file 1), as a complementary source, but are not used as a primary source due to the lack of transparency of the sorting algorithm [26]. The results will be restricted to the first 200 results.

National fruit research institutes in Germany (INRES), France (CTIFL), Italy (Laimburg Research Centre) and Spain (IRTA) were contacted to inquire about their relevant publications. The provided papers will undergo an eligibility check and will be listed accordingly as additional material.

Further, selected relevant institutional websites are searched for using simplified keywords in English, German, French, Dutch and Spanish (Table 4). This list may increase during the review process.

Article screening process

The screening process will be conducted by the authors of the protocol. In accordance with the CEE guidelines [24], several screening steps can be distinguished and the number of eliminated studies at every step of the screening process will be reported in a ROSES flow diagram using a dedicated tool [27].

Firstly, all search results will be collected and duplicates removed using Endnote Desktop software [28] following the methodology by Bramer et al. [29].

Secondly, the title and abstract of every remaining article will be read. Articles will be removed if they do not fulfil one of the eligibility criteria (see Table 5). The online tool ‘Rayyan’ (https://rayyan.qcri.org/) is preferred for this step because it enables multiple users to assess papers and conflict solving. Every abstract will be read individually, but papers can be filtered by potential exclusion keywords (e.g. “mango”) which supports an efficient inclusion or exclusion of articles while recording the reason for exclusion, e.g. “wrong population” [30].

Table 5 Primary inclusion and exclusion criteria based on title and abstract
Full size table

For consistency checking, a random selection of 10% of the articles will be reviewed by more than one reviewer and a Kappa score [31] will be computed. If a lower agreement than 0.6 is reached, the criteria definition and examples in Table 5 will be rediscussed and adjusted. In case no consensus can be found, a third reviewer will be consulted for mediation. The resulting list of excluded articles and conflict resolution documentation will be made accessible online and added as supplementary material to the review.

Thirdly, the included references will be gathered by means of a reference manager software [28] and full texts will be retrieved. Corresponding authors will be asked for their copy if there is no institutional access (incl. interlibrary loan). Articles without full text access will be listed separately and attempts made to retrieve the full text will be documented.

Fourthly, publications will be read at full text level. In case its abstract led to the inclusion of a publication, but the full text reveals ineligibility, the exclusion and reason thereof, will be documented in a separate table as additional material. At this stage 10% of the full texts will be reviewed by more than one author and conflicts solved with the mediation of a third reviewer. The results will equally be reported.

Eligibility criteria

Articles will be retained or excluded based on the criteria listed in Table 5. Studies will be retained when they compare the effect of a given measure on the local temperature, the plant development (e.g. budburst delay), the condition/count of buds or flowers or yields against a control population. Articles that fail to meet one or more criteria will be excluded.

Study validity assessment

Studies in the environmental sciences rarely satisfy gold standards of internal and external validity simultaneously. Studies that do not satisfy certain criteria (e.g. min. sample size, random selection of experimental units) are considered of low validity for this study. This does not necessarily mean that they generally lack quality. Hence, existing critical appraisal tools for systematic reviews need to be adjusted for the context of environmental sciences [32]. Studies not reporting the necessary details have an uncertain validity which in this review will be treated as low validity.

To avoid a false sense of hierarchy, no ranking of study set-ups will be undertaken, unlike systematic reviews in the medical field, where pre-defined rankings exist [32]. Instead, the design of each included study will be assessed based on predefined quality criteria (Table 6). The first set of criteria is based on Bilotta et al. [32] where typical biases are listed regarding general aspects of the study design.

Table 6 Study bias assessment categories
Full size table

The second list of specific criteria was added in consultation with experts. Distinctive sections were designed for the judgement of sample sizes. In this review, the spatial application scale (experimental unit from which random sampling is possible) of an intervention can vary from an entire field (e.g. wind turbines) to a row (e.g. water sprinklers) or individual trees (e.g. foliar application), of which then trees or branches are sampled for analysis (observations). Considering the outcome measures, temporal replication at daily/nightly units within a field is possible if the outcomes are temperature measurements during or following an intervention, while in the case of analysing flower damage or yields repetitions of several days cannot be considered as independent treatments. Several crop cycles (years) are considered to be independent replicates. Considering the scale of the interventions, we differentiate between the application and assessment level, since not always all manipulated trees are assessed for damage.

All criteria formulations result from an interactive testing process with a subset of 20 papers which were independently assessed by two authors. This subset consisted of papers that were identified as likely eligible and directly accessible in the Web of Science Core Collection during the search string finetuning process. Ambiguous criteria were discussed and refined iteratively until there was a full consensus. The final criteria have been presented to the fruit research experts (PCFruit) and were approved with minor edits.

The validity assessment will be done on full texts of the eligible selection in a public project on the SysRev platform (https://sysrev.com). This software is preferred due to its efficient data extraction capabilities and is also suited for transparent conflict solving between multiple users [30]. For copyright reasons, full texts will not be uploaded.

Based on the criteria in Table 6, a final judgement of the study validity as low, middle or high will be made, without subjectively judging some criteria more important than others. If the study reports on methods and results in a rigorous way, at most one criteria may be classified as having a high risk of bias. These studies will be classified as overall low risk of bias. On the other hand, overall high risk of bias is assigned if the study has major shortcomings in conducting and/or presenting the research. This is defined as having a risk for bias for at least 3 criteria. All other studies will be classified as overall medium risk of bias. A detailed assessment of individual criteria will be reported in the supplementary materials of the review.

Data coding and extraction strategy

Within the project environment on sysrev.com, a set of pre-coded fields, as well as several open fields, will be established to extract details on the study in a systematic manner. Four categories of data will be extracted. The first collects metadata on the study including the setting of the experiment. Secondly, details on the population and the intervention will be recorded. Thirdly, the conditions for effectiveness will be retained when they are provided. Lastly, the results will be recorded with any given measure of variance. Table 7 shows the pre-coded answers or specifications for the open text fields. If applicable, several options can be selected at once, or written as free text (separated by a semicolon).

Table 7 Data extraction fields and codes filled with example answers to be pre-coded
Full size table

The definition of codes results from an iterative process. The repeatability of this process was tested before the finalisation of the protocol. Two authors extracted the data for the subset of 20 papers used for testing the validity assessment. When there was substantial disagreement, codes were adjusted until a Kappa score > 0.6 was reached. Furthermore, the codes were reviewed by experts from the research centre PCFruit.

During the data extraction phase, a random selection of studies (at least 10%) will be cross-checked by another author, with special attention to the outcome variables and human errors. The extracted data (tidy extraction sheets) will be available as additional material.

No Data/ Not applicable/Other is always an optional answer. Missing information will be reported during the synthesis stage. The extracted data records will be added as supplementary information to the review.

Potential effect modifiers/reasons for heterogeneity

To our best knowledge, no protocol exists for studies of the effectiveness of frost protection measures. Every fruit type has a specific sensitivity to temperatures which varies throughout the development stages [2].

Further, the variation in environmental conditions, the scale of the fields, the distance to the operated technology, the resolution, accuracy and position of the temperature measurement devices and setup in open fields or controlled environments (e.g. greenhouse or lab) make the comparison between studies difficult. Table 8 provides a list of the most important factors that may influence the effects. The list has been compiled based on experiences of the Frost Inno project by the Belgian research centre PCFruit [34].

Table 8 Effect modifiers
Full size table

As additional information to the review, the dominant soil textural class will be identified based on a global soil database (http://www.soilgrids.org) using the soilDB package in R [35]. When no data is available (urban areas), the nearest possible gridpoint will be retrieved. Data from the global Shuttle Radar Topography Mission [36] will be used for filling gaps regarding elevation.

Data synthesis and presentation

The research outputs will primarily be descriptive statistics of the identified publication and secondly a comparison of the performance of spring frost damage reduction strategies in temperate fruit orchards. The conditions for their effectiveness will be explained, to the extent that they are reported on in the selected literature.

A ROSES flow diagram will expose the body of literature found with the described search strings. The retained studies will be presented in an ‘evidence atlas’, as an interactive world map with datapoints comprising the collected metadata per study [37]. This ‘atlas’ will indicate the number of included studies, the studied fruit types over time etc. These findings will be contrasted with maps of global fruit production based on FAO statistics. Timelines on the level of detail in reporting and bar-plots of studies by set-up or environmental conditions will complement the ‘atlas’.

The narrative synthesis will be supported by limited quantitative analyses of the identified intervention effectiveness by outcome category. Accordingly, effect sizes will be computed separately for measures of (i) temperature, (ii) crop yields, (iii) damage to flowers or buds or (iv) delay/advance in phenology.

Based on extensive scoping exercises we anticipate very low levels of detail regarding the reported variability of the results and foresee working with Raw Mean Differences (RMD) for temperature, flower damage and budding delays.

The convention of the RMD is defined as Xintervention − Xcontrol, with X being the mean outcome measures per experiment. A positive RMD is an indication of an effective intervention in the case of temperature and bud break dates and of an ineffective/counterproductive one for the damages to flowers and buds. Comparisons between studies will be possible without standardisation since in all cases the scales are inherently meaningful (temperature in °C, and flower survival in %).

Studies reporting on final harvests (yields) as outcome measures (kg per tree or unit area) need to be pre-processed. First, all results will be converted to kg/tree, using the reported planting density or row distances. Still, these yields per tree cannot be compared directly between studies on different fruit types, given their various normal yield values. The Response Ratio (RR) will be computed instead as a relative measure of change independent of the order of magnitude [38]. The RR is computed as the Xintervention/Xcontrol, with X being the mean outcome measure per experiment. Values between 0 and 1 indicate a negative impact of the intervention on yields, a 1 indicates no effect and values > 1 indicate a positive effect of the intervention. Due to the asymmetry of the RR, a natural logarithmic transformation is used to make the scale symmetric [39].

Studies that fail to provide mean differences between populations in any of the outcome categories will be excluded from quantitative data synthesis.

We will further conduct linear mixed model regression of the effectiveness against reported characteristics of the fields and frost events. Since information on the precision is not available, this regression is an exploratory analysis only. Hypothesized contributing fixed factors include continuous variables like the recorded minimum temperature during the intervention, the elevation and latitude of the test sites and soil texture (given in % sand, clay or silt). Random factors include the study identifier, the fruit type and the development stage during which frost occurred.

Analysis of effectiveness and reported costs will be attempted, as this information is of high relevance for fruit farmers.

Sensitivity analysis to the study validity will be conducted, leaving out low validity studies, in order to test if low validity studies report higher or lower effect sizes.

All described analyses will be conducted in R language. The script will be provided as additional material.

Publication bias

As in all scientific domains, there is a higher likelihood to publish positive results than negative ones and studies with statistically significant findings are also more likely to be cited and by consequence to be found. Commonly funnel plots are used as means to express the publication bias. Without an estimation of variances only the sample sizes can be used for constructing these plots, acknowledging the limitations of this approach [40] .