Uncertainties in assessing the effect of climate change on agriculture using model simulation and uncertainty processing methods

Model simulation is an important way to study the effects of climate change on agriculture. Such assessment is subject to a range of uncertainties because of either incomplete knowledge or model technical uncertainties, impeding effective decision-making to climate change. On the basis of uncertainties in the impact assessment at different levels, this article systematically summarizes the sources and propagation of uncertainty in the assessment of the effect of climate change on agriculture in terms of the climate projection, the assessment process, and the crop models linking to climate models. Meanwhile, techniques and methods focusing on different levels and sources of uncertainty and uncertainty propagation are introduced, and shortcomings and insufficiencies in uncertainty processing are pointed out. Finally, in terms of how to accurately assess the effect of climate change on agriculture, improvements to further decrease potential uncertainty are suggested.

Global climate change is a current research hotspot, and clarifying its various effects is an important challenge for scientists [1,2]. There is no doubt that climate change will increase the fluctuation of food production and have an important effect on global food security [3]. On one hand, many studies have assessed crop response to climate change and the possible effects of future climate change on agricultural production [4][5][6]. On the other hand, studies have focused on the potential uncertainty in the effect of climate change, and reasonable mathematical methods have been employed [7][8][9][10]. Although it is difficult to prevent climate change and eliminate uncertainty in assessing its impact, the uncertainty can be minimized through greater awareness of sources of uncertainty and greater efforts to develop more *Corresponding author (email: zhangjh@cams.cma.gov.cn) sophisticated assessment tools.

Sources of uncertainty
Model simulation is an important tool for studying the present and future effects of climate fluctuations on agriculture. Because of the weak mechanism in a statistical model and limitations of extrapolation, the simulation of the effects of climate change on agriculture usually employ a climate model coupled with a mechanism-based crop growth model. Because any mathematical model is a simplification or approximation of reality, there is inevitable uncertainty. Walker et al. [11] summarized five types of simulation uncertainty for a mechanism-based model: (1) context uncertainty, (2) input uncertainty, (3) model structure uncertainty, (4) parameter uncertainty, and (5) modeling technical uncertainty. There is also much uncertainty in the simulation and assessment of climate change affecting agriculture in terms of the climate prediction, crop models and integrating climate models with crop models, and the uncertainty propagates through the assessment process.

Uncertainty in climate projection using a climate model
Climate projection based on a climate model differs from climate prediction [12]. It is almost impossible to forecast or predict weather conditions, population growth and economic development in the next 50 to 100 years with the current level of technology. Therefore, scientists have put forward the scenario concept [13]. Present predictions of future global climate change are usually based on one or more of the emission-scenario-driven climate model simulations; i.e. the results obtained only relate to a certain climate change scenario. There are large cognitive and technical uncertainties in the estimates of future climate change.
(i) Uncertainty in the emission scenario. The concentration scenarios retrieved from emission scenarios are basic input data of the climate model. The series of emission scenarios provided by the IPCC are the most widely used [14]. However, the IPCC series of greenhouse gas (GHG) emission scenarios have at least four types of uncertainty [15]: (1) uncertainty in the estimation of GHG emissions, (2) uncertainty in the decisions that governments will make relating to GHG emissions, (3) uncertainty in future technological advances and new energy development and use, which will affect GHG emissions, and (4) uncertainty stemming from there being no complete emission inventory reflecting the past and future statuses of GHG emissions. It is known that the IPCC Fifth Assessment Report will use representative pathways of emissions in its discussions. This kind of modeling considers population and economic growth, the evolution of technology, government policy, social institutions and other factors, and thus takes into account the feedback of human mitigation of climate change [16]. Nevertheless, the scenarios of future GHG emissions closely related to the population growth, a green economy, and political structure inevitably have unpredictable uncertainty; these will be the primary sources of uncertainty in any climate prediction.
(ii) Cognitive limitations of a climate system. Because of the limitations to our scientific understanding of various forces and physical processes in the current climate system, climate models still do not perfectly express cloud feedback, energy exchange between climate subsystems, sea ice and convection, and biofeedback and chemical processes [17]. Errors in simulating the Earth's radiative energy balance, clouds and precipitation are important sources of uncertainty in climate models [2]. Cloud feedback, heat uptake by the oceans, and the carbon cycle feedback mechanism are described differently in different climate models, and this increases uncertainty in predicting future climate [18].
(iii) Uncertainty in the climate model structure and parameters. Observation inaccuracies (including instrumental errors and interpolation errors resulting from discrete observations in space and time) and import errors in data analysis and assimilation give rise to uncertainty in the initial field of a numerical model. In other words, the meteorological data are only ever an approximation of the present actual atmospheric condition [19]. At the same time, the natural climate fluctuations occur on seasonal, yearto-year, and decadal scales, indicating potential unpredictability [18].

Uncertainty in the crop model
The crop model is a mechanism-based model based on crop physiology that dynamically simulates crop growth and yield formation processes, more accurately expresses the relationship between crop growth and changes in climate factors, and predicts crop yield under the specific conditions [20]. It is also the main tool recommended by the IPCC for quantitative investigation of the effects of climate change on crops. It is thus widely used in assessments of the regional effects of climate change [6]. However, due to the complex mechanism of crop physiology, some complex quantitative processes still only employ an experiential approach; hence, there is not yet a model suited to all ecology types. Moreover, in the assessment of the impact of climate change, no great importance is attached to model features, assumptions and limiting conditions, which increase the uncertainty in the impact assessment.
(i) Uncertainty in the crop response to climate change. With an increase in the CO 2 concentration and changes in temperature and precipitation, the environment for crop growth will be significant different from the current environment, and the crop response and the slow adaptation of crops to climate change are major uncertainties in current impact assessments. The effects of CO 2 fertilization and interaction between CO 2 and water use by crops are complex [21]. Some studies reported that previous model simulations may have overestimated the effect of an elevated CO 2 concentration on crops [22]. However, there remains disagreement on this issue [23]. Recent observations have shown that experiments overestimated the effect of rises in the CO 2 concentrations on the crop yield and underestimated the effect on biomass [24]. At the same time, because observations are often restricted to a limited number of test species and ecological zones, representative experiments should be validated on wider scales [24]. In addition, uncertainty in the critical temperatures of crops under climate warming is the main source of simulation error [5]. It has been shown that simulating crop growth on a long time scale using a crop model may underestimate the growing period and lead to low yield prediction [25]. Crops slowly adapt to climate change, and this response mechanism cannot be taken into account in current crop models.
(ii) Uncertainties in the crop model structure and procedure description. It has been shown that there are notable differences in the responses of different mechanisms described in crop models to changes in temperature [26][27][28][29], CO 2 concentration [28,30,31], and saturation vapor pressure [29] and climate model data of different resolution [32], and these differences tend to affect the assessment of climate change based on the specific models. Model complexity is a more controversial issue because it involves the establishment of different models. More importantly, the crop system is more complex than the model system description. Improving the model complexity can enhance the simulation, but it inevitably increases the difficulty of measuring parameters [33]. A simple model can greatly reduce the risk of excessive fitting because of complex model parameters. Currently, the development of a large-scale crop model is based on this idea of simplicity; however, a model with higher temporal resolution or a stronger mechanism would perhaps better reflect the mechanisms of the crop response to climate factors; e.g. simulating the effects of extreme weather events on crops requires this kind of model. In the assessment of the climate change impact, the choice of the type of model structure and level of complexity is critical; e.g. the EPIC model requires 60 input parameters [32], with less than 10% being meteorological parameters whereas the CropSyst model input parameters are relatively few [26], the CERES model is more sensitive to the soil water deficit [34], and the APSIM model is relatively sensitive to physical and chemical characteristics of the soil [35].
(iii) Uncertainties in model calibration and validation. Using crop models involves the localization and calibration of parameters, and model testing ensures the reliability of the simulation results. Because a crop model is established under certain climatic conditions, it is necessary to strictly examine the applicability of the crop model in terms of future significant changes in climate conditions [36][37][38]. Because the large number of models currently available still have to consider the effects of pests, weeds and extreme weather events on crops, the comparison of simulation results with measurements may reveal biases [5,39,40]. At the same time, comparing simulation results with measurements is generally based on a linear regression model, whose premise is the use of independently measured data sequences; in practice, multi-year measurements for various regions might not meet this assumption, giving rise to critical errors in calibration and validation [41]. The intercomparison of long sequences of crop production requires the elimination of other non-climatic factors contributing to the yield; i.e. technological progress. However, there is no satisfactory method of extracting the "climatic yield"; general filter methods such as those employing the moving average and orthogonal polynomials may lose information on the effect of climatic factors that have trend changes [42,43]. Furthermore, errors in crop, soil and weather data are an important source of uncertainty in model simulation [44,45].

Uncertainty in climate models linked with crop models
Scale differences between the climate model output and crop model input are a main source of uncertainty in the assessment of the effect of climate change on agriculture [46]. Typically, simulation of the effect of climate change on crops is based on certain methods for the downscaling to the general circulation model (GCM) output to match the crop model [47]. A simulation is conducted at the site-scale and then integrated into a regional impact assessment. Recent studies have indicated that the effects of soil texture and management measures are comparatively obvious in site-scale simulation and that the influence of climate factors is more obvious at the regional scale [48,49]. Model up-scaling can reduce the complexity of the model and significantly reduce the input of parameters [50]. Many researchers have developed large-scale plant-growth simulation models; e.g. GLAM [51,52], MCWLA [53], PRYSBI [54], and M-GAEZ [55]. These models were scaled-up by simplifying the biological physics, which significantly reduced input variables and parameters, and to a certain extent, took into account the effect of exceptional climatic events on crop growth stress and disease. However, in the above large-scale crop-yield models, the deviations of the simulation and actual yield are revised through a correction factor based on experience, and different plant varieties and management procedures need to be merged, which inevitably introduces new uncertainties [56]. At the same time, there are significant differences in the CO 2 response between large-scale simulation based on the canopy and site-scale simulation based on leaves [57].

Propagation of uncertainty
Not only is there uncertainty at multiple levels in the simulation of the effect of climate change on agriculture [58], but the uncertainty propagates from the top to the bottom layer in an in-depth assessment ( Figure 1) [59].
First, there is great uncertainty in the construction of GHG emission scenarios resulting from uncertainty in future socio-economic development. Additionally, there is uncertainty in our current understanding of carbon sources and sinks, further increasing the error in estimating atmospheric GHG concentrations. Furthermore, there is limited understanding of the extent that GHG force the climate system. The main physical processes are not yet fully described in global climate models, and the causes of future global climate change cannot be accurately estimated. Simulations of regional climate change are based on a global climate model owing to the complexity of the regional response to climate change and uncertainty in diversity and downscaling, thus making regional climate change predictions more unreliable. Finally, when assessing the local effect of climate change on agriculture using climate change scenarios of the climate model output, the resulting uncertainty in the impact assessment is maximized because of uncertainties in the crop biophysical responses and the model. Studies have shown that among numerous uncertainties in the assessment of the impact of climate change, uncertainty in climate prediction is the main source [10,60]. Furthermore, in assessing the crop response to climate change, the uncertainty in the crop response to temperature change is significantly higher than that to precipitation change [61].

Uncertainty processing methods for impact assessment
Climate change projection is the basis for assessing the future effect of climate change on agricultural and natural ecosystems [62]. Great efforts have been made to deal with uncertainty associated with the projection of future climate change, including developing emissions scenarios that are more scientific and reasonable, reducing the uncertainty in estimating GHG concentrations in the atmosphere through better understanding of the carbon cycle, strengthening basic research on how carbon cycles, coupling land surface processes and biogeochemical processes with climate models, and using multi-model ensembles in climate projections. The study of uncertainty in climate change prediction is far more broad and detailed than that of uncertainty in impact assessment. However, an important aspect of the assessment of impact uncertainty is the rational use of various emissions scenarios and climate projection scenarios derived from climate models based on the present climate projection capacity, choosing appropriate assessment scales and models, and scientifically characterizing uncertainty in the resulting outputs. In recent years, several important technologies and approaches have stood out, including sensitivity analysis, model comparison, ensemble modeling, and risk analysis based on Bayesian theory.

Sensitivity analysis
Sensitivity analysis is an important method for assessing how sensitive the output of a model is to parameter errors of the model [63]. Traditional sensitivity analysis is based on sensitivity analysis of a single variable. For example, Challinor et al. [10] analyzed the sensitivity of both climate and crop models to parameters, changing one parameter at a time in local sensitivity analysis. In many other model simulations of sensitivity analysis, sensitivity analysis of single variables is indispensable [29,54,57,59,64,65]. Yet this type of sensitivity analysis only addresses local parameter sensitivity and is unable to account for interactions between parameters; thus, it is not suited to highly nonlinear models [66]. Considering high parameter correlation in models, recently developed multivariate and global sensitivity analyses allow the simultaneous testing of multiple parameter changes and can analyze direct and indirect effects of each parameter, the advantages of which are well illustrated in the sensitivity analysis of crop model parameters [67,68]. Considering that previous sensitivity analyses were only concerned with the effect on the final output results, it has been recently suggested that it is necessary to perform a multivariate time series of the sensitivity analysis for the daily output of the results obtained from a discontinuous-step simulation model [69], and it is important to reduce the uncertainty in the model parameter estimation. It is noted that, if we consider the model structure of uncertainty based on a single model of sensitivity analysis, possible errors in the model structure of uncertainty will propagate to the assessment of parameter uncertainty [70].

Model comparison
Model comparison is an important way to assess structural uncertainties in models. For example, when simulating the impact of a doubled CO 2 concentration on rice production in South East Asia, Matthews et al. [31] employed three GCMs and two crop models, showing significant differences between climate scenarios derived with different climate models and also different yield variations derived with the different crop models. Mearns et al. [32] used the CERES and EPIC crop models and evaluated the effect of climate change on wheat and maize yields on the central plains of the United States, and found that simulations of the crop yields significantly differed for different spatial resolutions of the climate model. Through model comparison, Challinor et al. [57] further recognized that scale (leaf versus canopy), model calibration, and model complexity affect the simulation of the interaction between CO 2 and water stress. Furthermore, model comparison among empirical, statistical and mechanistic models is a method of intercomparison.
Peng et al. [71] examined the relationship between yield and minimum temperature using an empirical statistical model and suggested that yield declined by 10% with a nighttime temperature increase of 1°C under global warming, indicating that mechanistic crop models may underestimate the effects of temperature on yield. Sheehy et al. [72] tested this hypothesis using a mechanistic model and empirical model and the same weather datasets used in the study of Peng et al., and concluded that yield declines calculated by regression were overestimated mainly because of negative correlation between solar radiation and minimum temperature, and thus, mechanistic crop models were adequate for examining the temperature responses of crops. Further efforts were made by Lobell et al. [73] to examine the mechanisms behind the discrepancy between the predictions of an empirical model and mechanistic model, by considering the regions characterized by low correlations between minimum temperature and maximum temperature. The result indicated that the mechanistic crop model (CERES) overestimated the sensitivity of the grain filling rate to temperature, with higher minimum temperature increasing the harvest indices, and therefore moderated the adverse effect of the minimum temperature; this indicates considerable uncertainty in the temperature response of the grain filling rate in the CERES model.

Ensemble modeling
Ensemble modeling obtains an ensemble forecast result by controlling multiple models or single-mode parameter changes [74]. For example, throughout the fourth assessment report, the IPCC adopted several dozen models in considering uncertainty in the climate model structure [6,15]. In identifying global food security and fragile regions in the 2030s, Lobell et al. [75] used 20 state-of-the-art GCMs and considered 60 climate scenarios. In more recent research, Masutomi et al. [55] comprehensively considered the process/parameter uncertainty in climate projection, using 49 future climate projections of GCMs, in an assessment of the effect of climate change on rice production in Asia. Since Murphy et al. [8] constructed 53 perturbed physics simulations, producing a probabilistic climate scenario, probabilistic estimates of the effect of climate change on crops have been carried out in the past few years [76]. A noticeable case is the study of Challinor et al. [77], who presented ensemble yield simulations generated by perturbing parameters of GCMs and crop parameters, quantifying both climate and crop model uncertainty together with their relative magnitudes, and thus made a significant step for-ward in comprehensively estimating uncertainty in the assessment of the climate change impact.
As discussed above, a distinct characteristic of ensemble modeling is that prediction outputs are represented by an estimated confidence interval or in a probabilistic format rather than as deterministic values, and thus uncertainty can be quantified in a probabilistic manner.

Uncertainty processing based on risk assessment
The strategies discussed above only separately treat uncertainty of one type or limited sources in impact modeling and assessment, and thus they are too simplistic to satisfy a thorough investigation of uncertainty. Risk assessment, however, can be employed in the assessment of the climate change impact to manage as much uncertainty as possible. At the same time, the propagation of uncertainty can be appropriately estimated. In a risk assessment framework, an indispensable input is the probabilistic climate scenario, which can be directly used in risk assessment. Furthermore, an "end-to-end impact assessment" can be performed on the basis of risk [78,79]. Generally, for a more comprehensive consideration of uncertainty, Bayesian hierarchical modeling can be employed to construct probabilistic climate scenarios based on Monte Carlo simulation [59,64,80]. An example of such an approach was given by Luo et al. [81], who estimated the effect of climate change on wheat in South Australia in a risk-assessment framework, comprehensively considering the predictive uncertainty from GHG emission scenarios, the GCM structure, climate sensitivity, and the regional climate response. Moreover, another distinct advantage of risk assessment is that the gap between impact assessment and decision making can be bridged in a risk form, thus advancing plans for adaptation.

Uncertainty processing in crop simulation on different scales
Uncertainty in crop simulation can be reduced by comparing simulation results with in situ observations on a site scale. If the site-scale crop model is used on a regional scale for assessing the effects of climate change on agricultural production, then according to certain regional range of crop varieties and the merging of management parameters that cause uncertainty, the merging plus sampling simulation approach has yielded better results [82]. Because there is scarce regional information on crop varieties and management parameters in the case of large spatial heterogeneity, consideration of climate warming in a more systematic crop district would reduce the uncertainty. Crop and soil information in regional modeling can be obtained through intensive observation campaigns, validation through remote sensing and optimizing the results. Taking advantage of a wide range of experiments, the model genetic parameters can be calibrated, and combined with detailed regional genetic parameters, they can reduce uncertainty in genetic parameters on a regional scale [83]. In addition, on the basis of regional statistics and the site crop model, a GCM grid corresponding to the crop model can be developed to simplify and optimize the model for the site and allow the crop model to be combined with the atmospheric model [84], which is also a method of treating uncertainty in a regional-scale crop simulation. To reduce uncertainty derived from the spatial and temporal distributions of rainfall in crop yield simulation at a regional scale, de Wit et al. [85] used an ensemble Kalman filter (EnKF) to assimilate coarse-resolution satellite microwave sensor estimates of soil moisture and correct errors in the water balance of the WOFOST crop model. In their study, a probabilistic framework of crop modeling was first established using an ensemble approach, and then an EnKF was integrated into the probabilistic framework; meanwhile, model uncertainty was assessed by sampling from probability distributions of crop parameter values and by providing ensembles of meteorological forcing including uncertainty.

Other methodologies for uncertainty processing
To overcome inaccuracies in predicting regional precipitation using a GCM model, Ines et al. [86] developed a statistical bias-correction model based on the historical daily weather record, and applied the model to crop simulation. It was found that the yield simulation significantly improved after correction of the bias in the precipitation data. Baigorria [87] also applied the bias-correction method to process precipitation, temperature and radiation variables of RCM hindcast data, suggesting that bias-correcting all meteorological variables used by the crop model increased the predictability compared with the case of the individual bias-correction of rainfall. Such a method is promising in terms of better crop modeling when linking GCMs outputs to crop models, as the local climate predictability is obviously higher, by taking full advantage of statistical characteristics of the historical climate variables.
Generalized likelihood uncertainty estimation (GLUE) has been extensively applied in hydrology modeling to estimate the uncertainty associated with model outputs and parameter simulation [88]. The GLUE rejects the idea of an optimum parameter set in favor of the concept of equifinality, and hence, a set of behavioral models is selected where each model has certain likelihood, based upon which uncertainty can be assessed. The emergence of the GLUE in recent years has addressed uncertainty in crop modeling. For example, Mo et al. [89] applied the GLUE in analyzing parameter sensitivities and uncertainty bounds based on a simple soil-vegetation-atmosphere transfer model, and the performances of two-source and three-source models were compared. In essence, the GLUE is a type of ensemble modeling method and cannot only be applied to parameter estimation or uncertainty analysis but also allow analysis of uncertainty propagation within Bayesian estimation [90]. It is noted that when using the GLUE to estimate crop model parameters, different likelihood functions and methods of combing likelihood values may provide different estimations of parameters [91].

Current limitations in uncertainty processing
The processing of uncertainties is still in its development phase, and existing processing methods are limited in terms of both reducing uncertainty and quantifying uncertainty. Sensitivity analysis can only be carried out for parameter uncertainty examination for a specific model structure [92]. Model comparison, as a diagnostic method, does not directly reduce uncertainty and at the same time, is subject to restriction of the required data and number of available models. Stochastic theories have greatly advanced ensemble simulation instead of deterministic simulation; however, on the basis of a single model construction and through parameter perturbations, ensemble simulation may give rise to systemic bias because of uncertainty in the description of the model mechanism.
However, ensemble simulation based on multiple models bring about credibility limited of estimated results due to subjectivity of ensemble members selection and the deviation of performance evaluation among ensemble members. Although uncertainty processing in risk assessment can simultaneously deal with uncertainty at all levels, it only provides the system risks on the basis of partly considering uncertainty and make adaptation decisions totally dependent on the result may be misleading. At the same time, risk assessment is currently applied more in the uncertainty processing of weather prediction, and no uncertainty study of the integrated impact assessment model has been reported. Therefore, in terms of the tools and methods available for dealing with all uncertainty in the simulation of the effect of climate change on crops, the more powerful integrated processing tools and methods have yet to be fully explored and developed.

Future studies on the assessment of the effect of climate change on agriculture
Compared with overseas research, the processing of uncertainty in the assessment of the effect of climate change on agriculture is undeveloped in China. To further improve the reliability of the results of impact assessment and provide an effective scientific basis for decision-making, it is recommended that the following be noted in future assessments.
(1) Sources of uncertainty should be identified and projected completely before the assessment of performance; e.g. an uncertainty matrix can be used to qualitatively order the importance of uncertainties.
(2) In performance assessment, necessary quality control procedures should be first carried out for input data from different sources. The selected models should be rigorously tested. In addition, it is possible that multiple models should be used, and through model intercomparison, deficiencies in the model should be determined and necessary improvements and updates made. An appropriate assessment scale should be considered to minimize the uncertainty in linking with other models. Uncertainty analysis should play a diagnostic role in the assessment of each step of the implementation and the whole evaluation process.
(3) Finally, an integrated assessment should be carried out for general uncertainty in the assessment at the same time. It is necessary to make an objective explanation of the use of assessment methods and limitations of the unitization model, and inevitable uncertainties should be quantified and expressed. The specification of evaluation process is the premise for ensuring reliable results of assessment. However, improving the accuracy of climate change prediction and developing and improving an effective assessment tool are essential for reducing uncertainty in the evaluation results of the effects of climate change on agriculture. The forcing and physical processes of the climate system and crop responses to climate change also need to be further explored and clarified.
We thank two reviewers for valuable suggestions that improved this article.