Abstract
Both reliability and independence of global climate model (GCM) simulation are essential for model selection to generate a reasonable uncertainty range of dynamical downscaling simulations. In this study, we evaluate the performance and interdependency of 37 GCMs from the Coupled Model Intercomparison Project Phase 6 (CMIP6) in terms of seven key large-scale driving fields over 14 CORDEX domains. A multivariable integrated evaluation method is used to evaluate and rank the models’ ability to simulate multiple variables in terms of their climatological mean and interannual variability. The results suggest that the model performance varies considerably with seasons, domains, and variables evaluated, and no model outperforms in all aspects. However, the multi-model ensemble mean performs much better than almost all models. Among 37 CMIP6 models, the MPI-ESM1-2-HR and FIO-ESM-2-0 rank top two due to their overall good performance across all domains. To measure the model interdependency in terms of multiple fields, we define the similarity of multivariate error fields between pairwise models. Our results indicate that the dependence exists between most of the CMIP6 models, and the models sharing the same idea or/and concept generally show less independence. Furthermore, we hierarchically cluster the top 15 models with good performance based on the similarity of multivariate error fields to identify relatively independent models. Our evaluation can provide useful guidance on the selection of CMIP6 models based on their performance and relative independence, which helps to generate a more reliable ensemble of dynamical downscaling simulations with reasonable inter-model spread.
Similar content being viewed by others
Availability of data and material
The data used in this study is available for open access.
Code Availability
Not applicable.
References
Bands S, Herrera S, Fernandez J, Gutierrez JM (2013) How well do CMIP5 Earth System Models simulate present climate conditions in Europe and Africa? Clim Dyn 41:803–817. https://doi.org/10.1007/s00382-013-1742-8
Bi D, Dix M, Marsland S, O’Farrell S, Sullivan A, Bodman R et al (2020) Configuration and spin-up of ACCESS-CM2, the new generation Australian community climate and earth system simulator coupled model. J South Hemisph Earth Syst Sci 70(1):225–251. https://doi.org/10.1071/ES19040
Bishop CH, Abramowitz G (2013) Climate model dependence and the replicate Earth paradigm. Clim Dyn 41:885–900. https://doi.org/10.1007/s00382-012-1610-y
Brient F (2020) Reducing Uncertainties in Climate Projections with Emergent Constraints: Concepts, Examples and Prospects. Adv Atmos Sci 37:1–15. https://doi.org/10.1007/s00376-019-9140-8
Brunner L, Pendergrass AG, Lehner F, Merrifield AL, Lorenz R, Knutti R (2020) Reduced global warming from CMIP6 projections when weighting models by performance and independence. Earth Syst Dynam 11:995–1012. https://doi.org/10.5194/esd-11-995-2020
Buontempo C, Mathison C, Jones R, Willias K, Wang C, cSweeney C (2015) An ensemble climate projection for Africa. Clim Dyn 44:2097–2118. https://doi.org/10.1007/s00382-014-2286-2
Burrows SM, Maltrud M, Yang X, Zhu Q, Jeffery N, Shi X et al (2020) The DOE E3SM v1.1 biogeochemistry configuration: Description and simulated ecosystem-climate responses to historical changes in forcing. J Adv Model Earth Syst 12:e2019MS001766. https://doi.org/10.1029/2019MS001766
Collins M, Booth BBB, Bhaskaran B, Harris GR, Murphy JM, Sexton DMH, Webb MJ (2011) Climate model errors, feedbacks and forcings: a comparison of perturbed physics and multi-model ensemble. Clim Dyn 36:1737–1766. https://doi.org/10.1007/s00382-010-0808-0
Cox PM, Huntingford C, Williammson MS (2018) Emergent constraint on equilibrium climate sensitivity from global temperature variability. Nature 553:319–322. https://doi.org/10.1038/nature25450
Dai A, Rasmussen RM, Ikeda K, Liu C (2020) A new approach to construct representative future forcing data for dynamic downscaling. Clim Dyn 55:315–323. https://doi.org/10.1007/s00382-017-3708-8
Danabasoglu G, Lamarque J-F, Bacmeister J, Bailey DA, DuVivier AK, Edwards J et al (2020) The Community Earth System Model Version 2 (CESM2). J Adv Model Earth Syst 12. https://doi.org/10.1029/2019MS001916. e2019MS001916
Döscher R, Acosta M, Alessandri A, Anthoni P, Arneth A, Arsouze T et al (2021) The EC-Earth3 Earth System Model for the Climate Model Intercomparison Project 6. https://doi.org/10.5194/gmd-2020-446. Geosci Model Dev Discuss
Dosio A, Panitz H-J, Schubert-Frisius M, Lüthi D (2015) Dynamical downscaling of CMIP5 global circulation models over CORDEX-Africa with COSMO-CLM: evaluation over the present climate and analysis of the added value. Clim Dyn 44:2637–2661. https://doi.org/10.1007/s00382-014-2262-x
Elguindi N, Giorgi F, Turuncoglu U (2014) Assessment of CMIP5 global model simulations over the subset of CORDEX domains used in the Phase I CREMA. Clim Change 125:7–21. https://doi.org/10.1007/s10584-013-0935-9
Eyring V, Bony S, Meehl GA, Senior CA, Stevens B, Stouffer RJ, Taylor KE (2016) Overview of the Coupled Model Intercomparison Project Phase 6 (CMIP6) experimental design and organization. Geosci Model Dev 9:1937–1958. https://doi.org/10.5194/gmd-9-1937-2016
Giorgi F (2006) Regional climate modeling: status and perspectives. J Phys IV France 139:101–118. https://doi.org/10.1051/jp4:2006139008
Giorgi F, Gutowski WJ (2016) Coordinated Experiments for Projections of Regional Climate Change. Curr Clim Change Rep 2:202–210. https://doi.org/10.1007/s40641-016-0046-6
Giorgi F, Jones C, Asrar GR (2009) Addressing climate information needs at the regional level: the CORDEX framework. WMO Bull 58(3):175–183
Gutjahr O, Putrasahan D, Lohmann K, Jungclaus JH, Storch J-S, Brüggemann N, Haak H, Stössel A (2019) Max Planck Institute Earth System Model (MPI-ESM1.2) for the High-Resolution Model Intercomparison Project (HighResMIP). Geosci Model Dev 12:3241–3281. https://doi.org/10.5194/gmd-12-3241-2019
Gutowski WJ, Giorgi F, Timbal B, Frigon A, Jacob D, Kang H-S, Raghavan K, Lee B, Lennard C, Nikulin G, O’Rourke E, Rixen M, Solman S, Stephenson T, Tangang F (2016) WCRP COordinated Regional Downscaling EXperiment (CORDEX): a diagnostic MIP for CMIP6. Geosci Model Dev 9:4087–4095. https://doi.org/10.5194/gmd-9-4087-2016
Han Y, Zhang M-Z, Xu Z, Guo W (2021) Assessing the performance of 33 CMIP6 models in simulating the large-scale environmental fields of tropical cyclones. Clim Dyn. https://doi.org/10.1007/s00382-021-05986-4
Herger N, Abramowitz G, Knutti R, Angélil O, Lehmann K, Sanderson BM (2018) Selecting a climate model subset to optimise key ensemble properties. Earth Syst Dynam 9:135–151. https://doi.org/10.5194/esd-9-135-2018
Huang F, Xu Z, Guo W (2019) Evaluating vector winds in the Asian-Australian monsoon region simulated by 37 CMIP5 models. Clim Dyn 53:491–507. https://doi.org/10.1007/s00382-018-4599-z
Huang F, Xu Z, Guo W (2020) The linkage between CMIP5 climate models’ abilities to simulate precipitation and vector winds. Clim Dyn 54:4953–4970. https://doi.org/10.1007/s00382-020-05259-6
Jun M, Knutti R, Nychka D (2008) Spatial analysis to quantify numerical model bias and dependence: how many climate models are there? J Am Stat Assoc 103:934–947. https://doi.org/10.1198/016214507000001265
Jury MW, Prein AF, Truhetz H, Gobiet A (2015) Evaluation of CMIP5 Models in the Context of Dynamical Downscaling over Europe. J Clim 28:5575–5582. https://doi.org/10.1175/JCLI-D-14-00430.1
Kebe I, Sylla MB, Omotosho JA, Nikiema PM, Gibba P, Giorgi F (2017) Impact of GCM boundary forcing on regional climate modeling of West African summer monsoon precipitation and circulation features. Clim Dyn 48:1503–1516. https://doi.org/10.1007/s00382-016-3156-x
Knutti R (2008) Why are climate models reproducing the observed global surface warming so well? Geophys Res Lett 40:1194–1199. https://doi.org/10.1002/grl.50256
Knutti R (2010a) The end of model democracy?: An editorial comment. Clim Change 102:395–404. https://doi.org/10.1007/s10584-010-9800-2
Knutti R, Masson D, Gerrelman A (2013) Climate model genealogy: Generation CMIP5 and how we got there. Geophys Res Lett 40:1194–1199. https://doi.org/10.1002/grl.50256
Knutti R, Meehl GA, Allen MR, Stainforth D (2006) Constraining climate sensitivity from the seasonal cycle in surface temperature. J Clim 19:4224–4233. https://doi.org/10.1175/JCLI3865.1
Knutti R, Furrer R, Tebaldi C, Cermak J, Meehl GA (2010b) Challenges in combining projections from multiple climate models. J Clim 23:2739–2758. https://doi.org/10.1175/2009JCLI3361.1
Knutti R, Sedláček J (2012) Robustness and uncertainties in the new CMIP5 climate model projections. Nat Clim Change 3:369–373. https://doi.org/10.1038/nclimate1716
Knutti R, Sedláček J, Sanderson BM, Lorenz R, Fischer EM, Eyring V (2017) A climate model projection weighting scheme accounting for performance and interdependence. Geophys Res Lett 44:1909–1918. https://doi.org/10.1002/2016GL072012
Lin Y, Huang X, Liang Y, Qin Y, Xu S, Huang W et al (2020) Community Integrated Earth System Model (CIESM): Description and evaluation. J Adv Model Earth Syst 12. https://doi.org/10.1029/2019MS002036. e2019MS002036
McSweeney CF, Jones RG, Lee RW, Rowell DP (2015) Selecting CMIP5 GCMs for downscaling over multiple regions. Clim Dyn 44:3237–3260. https://doi.org/10.1007/s00382-014-2418-8
Mendlik T, Gobiet A (2016) Selecting climate simulations for impact studies based on multivariate patterns of climate change. Clim Change 135:381–393. https://doi.org/10.1007/s10584-015-1582-0
Mishra SK, Sahany S, Salunke P (2018) CMIP5 vs. CORDEX over the Indian region: how much do we benefit from dynamical downscaling? Theor Appl Climatol 133:1133–1141. https://doi.org/10.1007/s00704-017-2237-z
Plavcová E, Kyselý J (2012) Atmospheric circulation in regional climate models over Central Europe: links to surface air temperature and the influence of driving data. Clim Dyn 39:1681–1695. https://doi.org/10.1007/s00382-011-1278-8
Qu X, Hall A (2013) On the persistent spread in snow-albedo feedback. Clim Dyn 42:69–81. https://doi.org/10.1007/s00382-013-1774-0
Ranjha R, Tjernstrom M, Svensson G, Semedo A (2016) Modelling coastal low-level wind-jets: does horizontal resolution matter? Meteorol Atmos Phys 128:263–278. https://doi.org/10.1007/s00703-015-0413-1
Reichler T, Kim J (2008) How Well Do Coupled Models Simulate Today’s Climate? Bull Am Meteorol Soc 89(3):303–311. https://doi.org/10. 1175/BAMS-89-3-303
Rocheta E, Evans JP, Sharma A (2020) Correcting lateral boundary biases in regional climate modeling: the effect of the relaxation zone. Clim Dyn 55:2511–2521. https://doi.org/10.1007/s00382-020-05393-1
Ruane AC, Teichmann C, Arnell NW, Carter TR, Ebi KL, Frieler K, Goodess CM, Hewitson B, Horton R, Kovats RS, Lotze HK, Mearns LO, Navarra A, Ojima DS, Riahi K, Rosenzweig C, Themessl M, Vincent K (2016) The vulnerability, impacts, adaptation and climate services advisory board (VIACS AB V1.0) contribution to CMIP6. Geosci Model Dev 9:3493–3515. https://doi.org/10.5194/gmd-9-3493-2016
Sanderson BM, Knutti R, Caldwell P (2015a) A Representative Democracy to Reduce Interdependency in a Multimodel Ensemble. J Clim 28:5171–5194. https://doi.org/10.1175/JCLI-D-14-00362.1
Sanderson BM, Knutti R, Caldwell P (2015b) Addressing Interdependency in a Multimodel Ensemble by Interpolation of Model Properties. J Clim 28:5150–5170. https://doi.org/10.1175/JCLI-D-14-00361.1
Semmler T, Danilov S, Gierz P, Goessling HF, Hegewald J, Hinrichs C et al (2020) Simulations for CMIP6 with the AWI climate model AWI-CM‐1‐1. J Adv Model Earth Syst 12. https://doi.org/10.1029/2019MS002009. e2019MS002009
Taylor KE (2001) Summarizing multiple aspects of model performance in a single diagram. J Geophys Res 106:7183–7192. https://doi.org/10.1029/2000JD900719
Tian B (2015) Spread of model climate sensitivity linked to double-Intertropical Convergence Zone bias. Geophys Res Lett 42:4133–4141. https://doi.org/10.1002/2015GL064119
Tian B, Dong X (2020) The double-ITCZ Bias in CMIP3, CMIP5 and CMIP6 models based on annual mean precipitation. Geophys Res Lett 47. https://doi.org/10.1029/2020GL087232. :e2020GL087232
Wilks D (2011) Statistical methods in the atmospheric sciences, 3rd edn. Academic Press, USA, pp 721–723
Wu W, Lynch AH, Rivers A (2005) Estimating the Uncertainty in a Regional Climate Model Related to Initial and Lateral Boundary Conditions. J Clim 18:917–933. https://doi.org/10.1175/JCLI-3293.1
Xu J, Gao Y, Chen D, Xiao L, Ou T (2017a) Evaluation of global climate models for downscaling applications centred over the Tibetan Plateau. Int J Climatol 37:657–671. https://doi.org/10.1002/joc.4731
Xu Z, Han Y, Fu C (2017b) Multivariable integrated evaluation of model performance with the vector field evaluation diagram. Geosci Model Dev 10:3805–3820. https://doi.org/10.5194/gmd-10-3805-2017b
Xu Z, Hou Z, Han Y, Guo W (2016) A diagram for evaluating multiple aspects of model performance in simulating vector fields. Geosci Model Dev 9:4365–4380. https://doi.org/10.5194/ gmd-9-4365-2016
Xu Z, Yang Z-L (2012) An Improved Dynamical Downscaling Method with GCM Bias Corrections and Its Validation with 30 Years of Climate Simulations. J Clim 25(18):6271–6286. https://doi.org/10.1175/JCLI-D-12-00005.1
Xu Z, Yang Z-L (2015) A new dynamical downscaling approach with GCM bias corrections and spectral nudging. J Geophys Res Atmos 120:3036–3084. https://doi.org/10.1002/2014JD022958
Zhang M-Z, Xu Z, Han Y, Guo W (2021) An improved multivariable integrated evaluation method and tool (MVIETool) v1.0 for multimodel intercomparison. Geosci Model Dev 14:3079–3094. https://doi.org/10.5194/gmd-14-3079-2021
Zhu Y-Y, Yang S (2020) Evaluation of CMIP6 for historical temperature and precipitation over the Tibetan Plateau and its comparison with CMIP5. Adv Clim Change Res 11(3):239–251. https://doi.org/10.1016/j.accre.2020.08.001
Acknowledgements
We thank the climate modeling groups involved in CMIP6 project for producing and making their model outputs available. The ERA5 Reanalysis data was provided from the website at https://www.ecmwf.int/en/forecasts/datasets/reanalysis-datasets/era5. Japanese 55-year reanalysis projects were carried out by the Japan Meteorological Agency. The precipitation datasets from Climate Prediction Center and the Global Precipitation Climatology Climate were provided from the websites at https://psl.noaa.gov/data/gridded/tables/precipitation.html. The study was supported jointly by the National Key Research and Development Program of China (2017YFA0603803) and the National Science Foundation of China (41675105, 42075170, 42075152). This work was also supported by the Jiangsu Collaborative Innovation Center for Climate Change.
Funding
The study was supported jointly by the National Key Research and Development Program of China (2017YFA0603803) and the National Science Foundation of China (41,675,105, 42,075,170, 42,075,152). This work was also supported by the Jiangsu Collaborative Innovation Center for Climate Change.
Author information
Authors and Affiliations
Contributions
Meng-Zhuo Zhang, Zhongfeng Xu, and Ying Han designed the study. Meng-Zhuo Zhang performed the analysis and led the writing of the paper. All authors discussed the results and commented on the paper.
Corresponding author
Ethics declarations
Conflicts of interest/Competing interests:
The authors declare no conflicts of interest.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Zhang, MZ., Xu, Z., Han, Y. et al. Evaluation of CMIP6 models toward dynamical downscaling over 14 CORDEX domains. Clim Dyn (2022). https://doi.org/10.1007/s00382-022-06355-5
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s00382-022-06355-5