Skip to main content

Advertisement

Springer Nature Link
Log in

Evaluation of methods for selecting climate models to simulate future hydrological change

  • Published:
Climatic Change Aims and scope Submit manuscript

Abstract

A challenge for climate impact studies is the selection of a limited number of climate model projections among the dozens that are typically available. Here, we examine the impacts of methods for climate model selection on projections of runoff change for five different watersheds across the conterminous USA. The results from an ensemble of 29 global climate models and 29 corresponding hydrological model simulations are compared with the results that would have been obtained by applying six different selection methods to the climate model data and using only the selected models to drive the hydrological model. We evaluate each selection method based on whether the runoff projections produced by the method meet the method’s objective and on whether the results are sensitive to the number of models chosen. The Katsavounidis–Kuo–Zhang (KKZ) method, which is intended to maximize the spread in the projected climate change, was the only method that met both criteria for suitability. Although the KKZ method generally performed well, the results from both it and the other methods varied somewhat unpredictably based on region and number of models chosen. This study shows that the methods and models used in similar top–down studies should be carefully chosen and that the results obtained should be interpreted with caution.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
€32.70 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (France)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  • Abramowitz G, Bishop CH (2015) Climate model dependence and the ensemble dependence transformation of CMIP projections. J Clim 28(6):2332–2348

    Google Scholar 

  • Abramowitz G, Herger N, Gutmann E, Hammerling D, Knutti R, Leduc M, Lorenz R, Pincus R, Schmidt GA (2019) ESD Reviews: Model dependence in multi-model climate ensembles: weighting, sub-selection and out-of-sample testing. Earth Syst Dyn 10(1):91–105

    Google Scholar 

  • Al Aamery N, Fox JF, Snyder M (2016) Evaluation of climate modeling factors impacting the variance of streamflow. J Hydrol 542:125–142

    Google Scholar 

  • Arnell NW, Gosling SN (2013) The impacts of climate change on river flow regimes at the global scale. J Hydrol 486:351–364

    Google Scholar 

  • Bastola S, Murphy C, Sweeney J (2011) The role of hydrological modelling uncertainties in climate change impact assessments of Irish river catchments. Adv Water Resour 34(5):562–576

    Google Scholar 

  • Bishop CH, Abramowitz G (2013) Climate model dependence and the replicate Earth paradigm. Clim Dyn 41:885–900

    Google Scholar 

  • Brekke LD, Dettinger MD, Maurer EP, Anderson M (2008) Significance of model credibility in estimating climate projection distributions for regional hydroclimatological risk assessments. Clim Chang 89:371–394

    Google Scholar 

  • Brekke L, Thrasher BL, Maurer EP, Pruitt T (2013) Downscaled CMIP3 and CMIP5 climate projections: release of downscaled CMIP5 climate projections, comparison with preceding information, and summary of user needs. Technical report, U.S. Department of the Interior, Bureau of Reclamation Technical Services Center, Denver

  • Brekke L, Wood A, Pruitt T (2014) Downscaled CMIP3 and CMIP5 climate and hydrology projections: release of hydrology projections, comparison with preceding information, and summary of user needs. Technical report, U.S. Department of the Interior, Bureau of Reclamation Technical Services Center, Denver

  • Cannon AJ (2015a) Selecting GCM scenarios that span the range of changes in a multimodel ensemble: application to CMIP5 climate extremes indices. J Clim 28 (3):1260–1267

    Google Scholar 

  • Cannon AJ, Sobie SR, Murdock TQ (2015b) Bias correction of GCM precipitation by quantile mapping: how well do methods preserve changes in quantiles and extremes?. J Clim 28(17):6938–6959

    Google Scholar 

  • Casajus N, Périé C, Logan T, Lambert M-C, de Blois S, Berteaux D (2016) An objective approach to select climate scenarios when projecting species distribution under climate change. PLoS ONE:11

  • Chai T, Draxler RR (2014) Root mean square error (RMSE) or mean absolute error (MAE)? – arguments against avoiding RMSE in the literature. Geosci Model Dev 7(3):1247–1250

    Google Scholar 

  • Chen J, Brissette FP, Lucas-Picher P (2016) Transferability of optimally-selected climate models in the quantification of climate change impacts on hydrology. Clim Dyn 47:3359–3372

    Google Scholar 

  • Chen J, Brissette FP, Lucas-Picher P, Caya D (2017) Impacts of weighting climate models for hydro-meteorological climate change studies. J Hydrol 549:534–546

    Google Scholar 

  • Frontier S (1976) ÉTude de la décroissance des valeurs propres dans une analyse en composantes principales: Comparaison avec le modèle du bâton brisé. J Exp Mar Biol Ecol 25:67–75

    Google Scholar 

  • Gibson JR, Najjar RG (2000) The response of Chesapeake Bay salinity to climate-induced changes in streamflow. Limnol Oceanogr 45(8):1764–1772

    Google Scholar 

  • Hagemann S, Chen C, Clark DB, Folwell S, Gosling SN, Haddeland I, Hanasaki N, Heinke J, Ludwig F, Voß F, Wiltshire AJ (2013) Climate change impact on available water resources obtained using multiple global climate and hydrology models. Earth Syst Dyn 4:129–144

    Google Scholar 

  • Hartigan JA, Wong MA (1979) Algorithm AS 136: a k-means clustering algorithm. J R Stat Soc Ser C (Appl Stat) 28(1):100–108

    Google Scholar 

  • Held IM, Soden BJ (2006) Robust responses of the hydrological cycle to global warming. J Clim 19(21):5686–5699

    Google Scholar 

  • Herger N, Abramowitz G, Knutti R, Angélil O, Lehmann K, Sanderson BM (2017) Selecting a climate model subset to optimise key ensemble properties. Earth Syst Dyn Discuss 2017:1–24

    Google Scholar 

  • Hirabayashi Y, Mahendran R, Koirala S, Konoshima L, Yamazaki D, Watanabe S, Kim H, Kanae S (2013) Global flood risk under climate change. Nat Clim Chang 3:816–821

    Google Scholar 

  • Holman IP, Allen DM, Cuthbert MO, Goderniaux P (2012) Towards best practice for assessing the impacts of climate change on groundwater. Hydrogeol J 20:1–4

    Google Scholar 

  • Houle D, Bouffard A, Duchesne L, Logan T, Harvey R (2012) Projections of future soil temperature and water content for three southern quebec forested sites. J Clim 25:7690–7701

    Google Scholar 

  • Hubert L, Arabie P (1985) Comparing partitions. J Classif 2:193–218

    Google Scholar 

  • Jackson DA (1993) Stopping rules in principal components analysis: a comparison of heuristical and statistical approaches. Ecol 74(8):2204–2214

    Google Scholar 

  • Johnson TE, Butcher JB, Parker A, Weaver CP (2012) Investigating the sensitivity of U.S. streamflow and water quality to climate change: U.S. EPA Global Change Research Program’s 20 Watersheds Project. J Water Resour Plan Manag 138 (5):453–464

    Google Scholar 

  • Justić D, Rabalais NN, Turner RE (2005) Coupling between climate variability and coastal eutrophication: evidence and outlook for the northern Gulf of Mexico. J Sea Res 54(1):25–35

    Google Scholar 

  • Katsavounidis I, Kuo C, Zhang Z (1994) A new initialization technique for generalized Lloyd iteration. IEEE Signal Process Lett 1(10):144–146

    Google Scholar 

  • Kaufmann L, Rousseeuw PJ (1990) Finding groups in data: an introduction to cluster analysis. Wiley, New York

    Google Scholar 

  • Kerkhoff C, Künsch HR, Schär C (2015) A Bayesian hierarchical model for heterogeneous RCM–GCM multimodel ensembles. J Clim 28(15):6249–6266

    Google Scholar 

  • Knutti R, Furrer R, Tebaldi C, Cermak J, Meehl GA (2010) Challenges in combining projections from multiple climate models. J Clim 23:2739–2758

    Google Scholar 

  • Knutti R, Masson D, Gettelman A (2013) Climate model genealogy: generation CMIP5 and how we got there. Geophys Res Lett 40:1194–1199

    Google Scholar 

  • Krzysztofowicz R (2001) The case for probabilistic forecasting in hydrology. J Hydrol 249(1-4):2–9

    Google Scholar 

  • Leduc M, Laprise R, De elía R, Šeparović L (2016) Is institutional democracy a good proxy for model independence?. J Clim 29:8301–8316

    Google Scholar 

  • Li H, Sheffield J, Wood EF (2010) Bias correction of monthly precipitation and temperature fields from Intergovernmental Panel on Climate Change AR4 models using equidistant quantile matching. J Geophys Res 115:D10101

    Google Scholar 

  • Liang X, Lettenmaier DP, Wood EF, Burges SJ (1994) A simple hydrologically based model of land-surface water and energy fluxes for general-circulation models. J Geophys Res Atmosph 99(D7):14415–14428

    Google Scholar 

  • Liang X, Wood EF, Lettenmaier DP (1996) Surface soil moisture parameterization of the VIC-2l model: evaluation and modification. Glob Planet Chang 13:195–206

    Google Scholar 

  • Maraun D (2013) Bias correction, quantile mapping, and downscaling: revisiting the inflation issue. J Clim 26(6):2137–2143

    Google Scholar 

  • Maraun D, Shepherd TG, Widmann M, Zappa G, Walton D, Gutiérrez JM, Hagemann S, Richter I, Soares PMM, Hall A, Mearns LO (2017) Towards process-informed bias correction of climate change simulations. Nat Clim Chang 7 (11):764–773

    Google Scholar 

  • Masson D, Knutti R (2011) Climate model genealogy. Geophys Res Lett:38

  • Maurer EP, Wood AW, Adam JC, Lettenmaier DP (2002) A long-term hydrologically based dataset of land surface fluxes and states for conterminous United States. J Clim 15:3237–3251

    Google Scholar 

  • Maurer EP, Pierce DW (2014) Bias correction can modify climate model simulated precipitation changes without adverse effect on the ensemble mean. Hydrol Earth Syst Sci 18(3):915–925

    Google Scholar 

  • McSweeney CF, Jones RG, Lee RW, Rowell DP (2015) Selecting CMIP5 GCMs for downscaling over multiple regions. Clim Dyn 44:3237–3260

    Google Scholar 

  • Melsen LA, Addor N, Mizukami N, Newman AJ, Torfs PJJF, Clark MP, Uijlenhoet R, Teuling AJ (2018) Mapping (dis)agreement in hydrologic projections. Hydrol Earth Syst Sci 22(3):1775–1791

    Google Scholar 

  • Mendlik T, Gobiet A (2016) Selecting climate simulations for impact studies based on multivariate patterns of climate change. Clim Chang 135:381–393

    Google Scholar 

  • Milligan GW, Cooper MC (1986) A study of the comparability of external criteria for hierarchical cluster analysis. Multivar Behav Res 21:441–485

    Google Scholar 

  • Milly P, Wetherald RT, Dunne KA, Delworth TL (2002) Increasing risk of great floods in a changing climate. Nature 415:514–517

    Google Scholar 

  • Najafi MR, Moradkhani H (2013) A hierarchical Bayesian approach for the analysis of climate change impact on runoff extremes. Hydrol Process 28(26):6292–6308

    Google Scholar 

  • Nijssen B, Lettenmaier DP, Liang X, Wetzel SW, Wood EF (1997) Streamflow simulation for continental-scale river basins. Water Resour Res 33(4):711–724

    Google Scholar 

  • Nijssen B, O’Donnell GM, Hamlet AF, Lettenmaier DP (2001) Hydrologic sensitivity of global rivers to climate change. Clim Chang 50:143–175

    Google Scholar 

  • Ott I, Duethmann D, Liebert J, Berg P, Feldmann H, Ihringer J, Kunstmann H, Merz B, Schaedler G, Wagner S (2013) High-resolution climate change impact analysis on medium-sized river catchments in Germany: an ensemble assessment. J Hydrometeorol 14:1175–1193

    Google Scholar 

  • Pierce DW, Cayan DR, Maurer EP, Abatzoglou JT, Hegewisch KC (2015) Improved bias correction techniques for hydrological simulations of climate change. J Hydrometeorol 16(6):2421–2442

    Google Scholar 

  • Rabalais NN, Turner RE, Diaz RJ, Justić D (2009) Global change and eutrophication of coastal waters. ICES J Mar Sci 66:1528–1537

    Google Scholar 

  • Ramos MH, van Andel SJ, Pappenberger F (2013) Do probabilistic forecasts lead to better decisions? Hydrol Earth Syst Sci 17(6):2219–2232

    Google Scholar 

  • Riahi K, Rao S, Krey V, Cho C, Chirkov V, Fischer G, Kindermann G, Nakicenovic N, Rafaj P (2011) RCP 8.5-a scenario of comparatively high greenhouse gas emissions. Clim Chang 109:33–57

    Google Scholar 

  • Rousseeuw PJ (1987) Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. J Comput Appl Math 20:53–65

    Google Scholar 

  • Sanderson BM, Knutti R, Caldwell P (2015) A representative democracy to reduce interdependency in a multimodel ensemble. J Clim 28:5171–5194

    Google Scholar 

  • Schewe J, Heinke J, Gerten D, Haddeland I, Arnell NW, Clark DB, Dankers R, Eisner S, Fekete BM, Colón-González FJ, Gosling SN, Kim H, Liu X, Masaki Y, Portmann FT, Satoh Y, Stacke T, Tang Q, Wada Y, Wisser D, Albrecht T, Frieler K, Piontek F, Warszawski L, Kabat P (2014) Multimodel assessment of water scarcity under climate change. Proc Natl Acad Sci 111(9):3245–3250

    Google Scholar 

  • Sriver RL, Forest CE, Keller K (2015) Effects of initial conditions uncertainty on regional climate variability: an analysis using a low-resolution CESM ensemble. Geophys Res Lett 42(13):5468–5476

    Google Scholar 

  • Steinschneider S, McCrary R, Mearns LO, Brown C (2015) The effects of climate model similarity on probabilistic climate projections and the implications for local, risk-based adaptation planning. Geophys Res Lett 42:5014–5022

    Google Scholar 

  • Tebaldi C, Knutti R (2007) The use of the multi-model ensemble in probabilistic climate projections. Philosophical Transactions of the Royal Society A: Mathematical. Phys Eng Sci 365:2053–2075

    Google Scholar 

  • Teng J, Vaze J, Chiew FHS, Wang B, Perraud J-M (2012) Estimating the relative uncertainties sourced from GCMs and hydrological models in modeling climate change impact on runoff. J Hydrometeorol 13:122–139

    Google Scholar 

  • Terando A, Keller K, Easterling WE (2012) Probabilistic projections of agro-climate indices in North America. Journal of Geophysical Research Atmospheres:117

  • Vetter T, Reinhardt J, Flörke M, Griensven A, Hattermann F, Huang S, Koch H, Pechlivanidis IG, Plötner S, Seidou O, Su B, Vervoort RW, Krysanova V (2017) Evaluation of sources of uncertainty in projected hydrological changes under climate change in 12 large-scale river basins. Clim Chang:141

  • Vicuna S, Maurer EP, Joyce B, Dracup JA, Purkey D (2007) The sensitivity of California water resources to climate change scenarios. J Amer Water Resour Assoc (JAWRA) 43(2):482–498

    Google Scholar 

  • Wang H-M, Chen J, Cannon AJ, Xu C-Y, Chen H (2018) Transferability of climate simulation uncertainty to hydrological climate change impacts. Hydrol Earth Syst Sci 22:3739–3759

    Google Scholar 

  • Weigel AP, Knutti R, Liniger MA, Appenzeller C (2010) Risks of model weighting in multimodel climate projections. J Clim 23:4175–4191

    Google Scholar 

  • Weiland FCS, van Beek LPH, Weerts AH, Bierkens MFP (2012) Extracting information from an ensemble of GCMs to reliably assess future global runoff change. J Hydrol 412-413:66–75

    Google Scholar 

  • Whetton P, Macadam I, Bathols J, O’Grady J (2007) Assessment of the use of current climate patterns to evaluate regional enhanced greenhouse response patterns of climate models. Geophys Res Lett:34

  • Wilby RL, Dessai S (2010) Robust adaptation to climate change. Weather 65 (7):180–185

    Google Scholar 

  • Wilcke RAI, Bärring L (2016) Selecting regional climate scenarios for impact modelling studies. Environ Modell Softw 78:191–201

    Google Scholar 

  • Willmott CJ, Matsuura K (2005) Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance. Clim Res 30:79–82

    Google Scholar 

  • Willmott CJ, Matsuura K, Robeson SM (2009) Ambiguities inherent in sums-of-squares-based error statistics. Atmos Environ 43(3):749–752

    Google Scholar 

  • Wood AW, Leung LR, Sridhar V, Lettenmaier DP (2004) Hydrologic implications of dynamical and statistical approaches to downscaling climate model outputs. Clim Chang 62:189–216

    Google Scholar 

  • Zubler EM, Fischer AM, Fröb F, Liniger MA (2016) Climate change signals of CMIP5 general circulation models over the Alps—impact of model selection. International Journal of Climatology:3088–3104

    Google Scholar 

Download references

Acknowledgments

We thank the anonymous reviewers and editor for comments that improved this manuscript.We also thank the World Climate Research Programme’s Working Group on Coupled Modelling, the U.S. Department of Energy’s Program for Climate Model Diagnosis and Intercomparison, the Global Organization for Earth System Science Portals, and the climate modeling groups (Table S1) for their roles in CMIP, as well as the providers of the downscaled hydrology projections.

Funding

Funding was provided by the National Science Foundation (CBET-1360286), Pennsylvania Sea Grant (NA10OAR4170063), and National Oceanic and Atmospheric Administration’s National Centers for Coastal Ocean Science (NA16NOS4780207 to Virginia Institute of Marine Science).

Author information

Author notes

  1. Andrew C. Ross

    Present address: Princeton University Program in Atmospheric and Oceanic Sciences, Princeton, NJ, USA

Authors and Affiliations

  1. Department of Meteorology and Atmospheric Science, The Pennsylvania State University, University Park, Pennsylvania, PA, USA

    Andrew C. Ross & Raymond G. Najjar

Authors
  1. Andrew C. Ross
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Raymond G. Najjar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andrew C. Ross.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

(PDF 69.2 KB)

(CSV 1.73 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ross, A.C., Najjar, R.G. Evaluation of methods for selecting climate models to simulate future hydrological change. Climatic Change 157, 407–428 (2019). https://doi.org/10.1007/s10584-019-02512-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10584-019-02512-8

Keywords

Search

Navigation