Abstract
The relative performance of global climate models (GCMs) of phases 5 and 6 of the coupled model intercomparison project (CMIP5 and CMIP6, respectively) was assessed in this study based on their ability to simulate annual and seasonal mean rainfall and temperature over Bangladesh for the period 1977–2005. Multiple statistical metrics were used to measure the performance of the GCMs at 30 meteorological observation stations. Two robust multi-criteria decision analysis methods were used to integrate the results obtained using different metrics for an unbiased ranking of the GCMs. The results revealed MIROC5 as the most skillful among CMIP5 GCMs and ACCESS-CM2 among CMIP6 GCMs. Overall, CMIP6 MME showed a significant improvement in simulating rainfall and temperature over Bangladesh compared to CMIP5 MME. The highest improvements were found in simulating cold season (winter and post-monsoon) rainfall and temperature in higher elevated areas. The improvement was relatively more for rainfall than for temperature. The models could capture the interannual variability of annual and seasonal rainfall and temperature reliably, except for the winter rainfall. However, systematic wet and cold/warm biases still exist in CMIP6 models for Bangladesh. CMIP6 GCMs showed higher spatial correlations with observed data, but the higher difference in standard deviations and centered root mean square errors compared to CMIP5 GCMs indicates better performance in simulating geographical distribution but lower performance in simulating spatial variability of most of the climate variables except for minimum temperature at different timescales. In terms of Taylor skill score, the CMIP6 MME showed higher performance in simulating rainfall but lower performance in simulating temperature than CMIP5 MME for most of the timeframes. The findings of this study suggest that the added value of rainfall and temperature simulations in CMIP6 models is not consistent among the climate models used in this research. However, it sets a precedent for future research on climate change risk assessment for the scientific community.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
Data used in this study are available from the first author upon request (milonbrri@gmail.com).
References
Ahmed K, Sachindra DA, Shahid S, Demirel MC, Chung ES (2019a) Selection of multi-model ensemble of general circulation models for the simulation of precipitation and maximum and minimum temperature based on spatial assessment metrics. Hydrol Earth Syst Sci 23:4803–4824. https://doi.org/10.5194/hess-23-4803-2019
Ahmed K, Shahid S, Sachindra DA, Nawaz N, Chung ES (2019b) Fidelity assessment of general circulation model simulated precipitation and temperature over Pakistan using a feature selection method. J Hydrol 573:281–298. https://doi.org/10.1016/j.jhydrol.2019.03.092
Ahmed K, Shahid S, Wang X, Nawaz N, Najeebullah K (2019c) Evaluation of gridded precipitation datasets over arid regions of Pakistan. Water (Switzerland) 11. https://doi.org/10.3390/w11020210
Ahmed K, Sachindra DA, Shahid S, Iqbal Z, Nawaz N, Khan N (2020) Multi-model ensemble predictions of precipitation and temperature using machine learning algorithms. Atmos Res 236:104806. https://doi.org/10.1016/j.atmosres.2019.104806
Almazroui M, Saeed S, Saeed F, Islam MN, Ismail M (2020) Projections of precipitation and temperature over the South Asian countries in CMIP6. Earth Systems and Environment 4:297–320. https://doi.org/10.1007/s41748-020-00157-7
Attogouinon A, Lawin AE, Deliège JF (2020) Evaluation of general circulation models over the upper oueme river basin in the republic of Benin. Hydrology. 7. https://doi.org/10.3390/hydrology7010011
Baker NC, Huang HP (2014) A comparative study of precipitation and evaporation between CMIP3 and CMIP5 climate model ensembles in semiarid regions. J Clim 27:3731–3749. https://doi.org/10.1175/JCLI-D-13-00398.1
Behar O, Khellaf A, Mohammedi K (2015) Comparison of solar radiation models and their validation under Algerian climate - the case of direct irradiance. Energy Convers Manag 98:236–251. https://doi.org/10.1016/j.enconman.2015.03.067
Chan SC, Kendon EJ, Fowler HJ, Blenkinsop S, Ferro CAT, Stephenson DB (2012) Does increasing the spatial resolution of a regional climate model improve the simulated daily precipitation? Clim Dyn 41:1475–1495. https://doi.org/10.1007/s00382-012-1568-9
Chen J, Brissette FP, Leconte R (2011) Uncertainty of downscaling method in quantifying the impact of climate change on hydrology. J Hydrol 401:190–202. https://doi.org/10.1016/j.jhydrol.2011.02.020
Chen X, Liu Y, Wu G (2017) Understanding the surface temperature cold bias in CMIP5 AGCMs over the Tibetan Plateau. Adv Atmos Sci 34:1447–1460. https://doi.org/10.1007/s00376-017-6326-9
Chowdhury MR (2003) The El Nino-Southern Oscillation (ENSO) and seasonal flooding—Bangladesh. Theor Appl Climatol 76(1-2):105–124
Despotovic M, Nedic V, Despotovic D, Cvetanovic S (2015) Review and statistical analysis of different global solar radiation sunshine models. Renew Sust Energ Rev 52:1869–1880
Duan A, Hu J, Xiao Z (2013) The Tibetan plateau summer monsoon in the CMIP5 simulations. J Clim 26:7747–7766. https://doi.org/10.1175/JCLI-D-12-00685.1
Eyring V, Bony S, Meehl GA, Senior CA, Stevens B, Stouffer RJ, Taylor KE (2016) Overview of the Coupled Model Intercomparison Project Phase 6 (CMIP6) experimental design and organization. Geosci Model Dev 9:1937–1958. https://doi.org/10.5194/gmd-9-1937-2016
Eyring V, Cox PM, Flato GM, Gleckler PJ, Abramowitz G, Caldwell P, Collins WD, Gier BK, Hall AD, Hoffman FM, Hurtt GC, Jahn A, Jones CD, Klein SA, Krasting JP, Kwiatkowski L, Lorenz R, Maloney E, Meehl GA, Pendergrass AG, Pincus R, Ruane AC, Russell JL, Sanderson BM, Santer BD, Sherwood SC, Simpson IR, Stouffer RJ, Williamson MS (2019) Taking climate model evaluation to the next level. Nat Clim Chang 9:102–110. https://doi.org/10.1038/s41558-018-0355-y
Foley AM (2010) Uncertainty in regional climate modelling: a review. Prog Phys Geogr 34:647–670. https://doi.org/10.1177/0309133310375654
Ghose B, Islam ARMT, Kamruzzaman M, Moniruzzaman M, Hu Z (2021) Climate-induced rice yield anomalies linked to large-scale atmospheric circulation in Bangladesh using multi-statistical modeling. Theor Appl Climatol 144:1077–1099. https://doi.org/10.1007/s00704-021-03584-2
Gleckler PJ, Taylor KE, Doutriaux C (2008) Performance metrics for climate models. J Geophys Res-Atmos 113. https://doi.org/10.1029/2007JD008972
Guo Y, Dong WJ, Ren FM et al (2013) Surface air temperature simulations over China with CMIP5 and CMIP3. Adv Clim Chang Res 4:145–152. https://doi.org/10.3724/SP.J.1248.2013.145
Gupta HV, Kling H, Yilmaz KK, Martinez GF (2009) Decomposition of the mean squared error and NSE performance criteria: implications for improving hydrological modelling. J Hydrol 377:80–91. https://doi.org/10.1016/j.jhydrol.2009.08.003
Gusain A, Ghosh S, Karmakar S (2020) Added value of CMIP6 over CMIP5 models in simulating Indian summer monsoon rainfall. Atmospheric Research 232. 232:104680. https://doi.org/10.1016/j.atmosres.2019.104680
Heinze C, Eyring V, Friedlingstein P, et al (2019) ESD reviews: climate feedbacks in the Earth system and prospects for their evaluation. Earth System Dynamics
Hijmans RJ, Cameron SE, Parra JL, Jones PG, Jarvis A (2005) Very high resolution interpolated climate surfaces for global land areas. Int J Climatol 25:1965–1978. https://doi.org/10.1002/joc.1276
IPCC (2013) Working Group I Contribution to the IPCC Fifth Assessment Report - Summary for Policymakers. Climate Change 2013: The Physical Science Basis. https://doi.org/10.1017/CBO9781107415324.004
Islam ARMT, Rahman MS, Khatun R, Hu Z (2020) Spatiotemporal trends in the frequency of daily rainfall in Bangladesh during 1975-2017. Theor Appl Climatol 141(3-4):869–887. https://doi.org/10.1007/s00704-020-03244-x
Jain S, Salunke P, Mishra SK, Sahany S (2019) Performance of CMIP5 models in the simulation of Indian summer monsoon. Theor Appl Climatol 137:1429–1447. https://doi.org/10.1007/s00704-018-2674-3
Johnson F, Westra S, Sharma A, Pitman AJ (2011) An assessment of GCM skill in simulating persistence across multiple time scales. J Clim 24:3609–3623. https://doi.org/10.1175/2011JCLI3732.1
Kamruzzaman M, Min-Won J, Syewoon H, Taeil J (2018) Evaluating the agricultural drought for Pre-Kharif season in Bangladesh using MODIS vegetation health index. Journal of the Korean Society of Agricultural Engineers 60:55–63. https://doi.org/10.5389/KSAE.2018.60.6.055
Kamruzzaman M, Jang MW, Cho J, Hwang S (2019a) Future changes in precipitation and drought characteristics over Bangladesh under CMIP5 climatological projections. Water (Switzerland) 11. https://doi.org/10.3390/w11112219
Kamruzzaman M, Hwang S, Cho J, Jang MW, Jeong H (2019b) Evaluating the spatiotemporal characteristics of agricultural drought in Bangladesh using effective drought index. Water (Switzerland) 11. https://doi.org/10.3390/W11122437
Kamworapan S, Surussavadee C (2019) Evaluation of CMIP5 global climate models for simulating climatological temperature and precipitation for southeast Asia. Adv Meteorol 2019:1–18. https://doi.org/10.1155/2019/1067365
Katzenberger A, Schewe J, Pongratz J, Levermann A (2021) Robust increase of Indian monsoon rainfall and its variability under future warming in CMIP6 models. Earth Syst Dynam 12:367–386. https://doi.org/10.5194/esd-12-367-2021
Khan N, Shahid S, Ahmed K, Ismail T, Nawaz N, Son M (2018a) Performance assessment of general circulation model in simulating daily precipitation and temperature using multiple gridded datasets. Water (Switzerland) 10. https://doi.org/10.3390/w10121793
Khan N, Shahid S, Ahmed K, Ismail T, Nawaz N, Son M (2018b) Performance assessment of general circulation model in simulating daily precipitation and temperature using multiple gridded datasets. Water (Switzerland) 10. https://doi.org/10.3390/w10121793
Khan MJU, Islam AKMS, Das MK, Mohammed K, Bala SK, Islam GMT (2019) Observed trends in climate extremes over Bangladesh from 1981 to 2010. Clim Res 77:45–61. https://doi.org/10.3354/cr01539
Kling H, Fuchs M, Paulin M (2012) Runoff conditions in the upper Danube basin under an ensemble of climate change scenarios. J Hydrol 424-425:264–277. https://doi.org/10.1016/j.jhydrol.2012.01.011
Koch J, Demirel MC, Stisen S (2018) The SPAtial EFficiency metric (SPAEF): multiple-component evaluation of spatial patterns for optimization of hydrological models. Geosci Model Dev 11:1873–1886. https://doi.org/10.5194/gmd-11-1873-2018
Lin CY, Tung CP (2017) Procedure for selecting GCM datasets for climate risk assessment. Terr Atmos Ocean Sci 28:43–55. https://doi.org/10.3319/TAO.2016.06.14.01(CCA
Lutz AF, ter Maat HW, Biemans H, Shrestha AB, Wester P, Immerzeel WW (2016) Selecting representative climate models for climate change impact studies: an advanced envelope-based selection approach. Int J Climatol 36:3988–4005. https://doi.org/10.1002/joc.4608
Maxino CC, McAvaney BJ, Pitman AJ, Perkins SE (2008) Ranking the AR4 climate models over the Murray-Darling Basin using simulated maximum temperature, minimum temperature and precipitation. Int J Climatol 28:1097–1112. https://doi.org/10.1002/joc.1612
McMahon TA, Peel MC, Karoly DJ (2015) Assessment of precipitation and temperature data from CMIP3 global climate models for hydrologic simulation. Hydrol Earth Syst Sci 19:361–377. https://doi.org/10.5194/hess-19-361-2015
Meehl GA, Covey C, Delworth T, Latif M, McAvaney B, Mitchell JFB, Stouffer RJ, Taylor KE (2007) The WCRP CMIP3 multimodel dataset: a new era in climatic change research. Bull Am Meteorol Soc 88:1383–1394. https://doi.org/10.1175/BAMS-88-9-1383
Mohsenipour M, Shahid S, Chung E sung, Wang X jun (2018) Changing pattern of droughts during cropping seasons of Bangladesh. Water Resour Manag https://doi.org/10.1007/s11269-017-1890-4, 32, 1555, 1568
Nashwan MS, Shahid S (2020) A novel framework for selecting general circulation models based on the spatial patterns of climate. Int J Climatol 40:4422–4443. https://doi.org/10.1002/joc.6465
Nashwan MS, Shahid S, Wang X (2019) Assessment of satellite-based precipitation measurement products over the hot desert climate of Egypt. Remote Sens 11. https://doi.org/10.3390/rs11050555
Nie S, Fu S, Cao W, Jia X (2020) Comparison of monthly air and land surface temperature extremes simulated using CMIP5 and CMIP6 versions of the Beijing Climate Center climate model. Theor Appl Climatol 140:487–502. https://doi.org/10.1007/s00704-020-03090-x
Northrop PJ (2013) Comments on “A simple, coherent framework for partitioning uncertainty in climate predictions.” Journal of Climate
O’Neill BC, Kriegler E, Riahi K, Ebi KL, Hallegatte S, Carter TR, Mathur R, van Vuuren DP (2014) A new scenario framework for climate change research: the concept of shared socioeconomic pathways. Clim Chang 122:387–400. https://doi.org/10.1007/s10584-013-0905-2
Ogata T, Ueda H, Inoue T et al (2014) Projected future changes in the Asian monsoon: a comparison of CMIP3 and CMIP5 model results. J Meteorol Soc Jpn 92:207–225. https://doi.org/10.2151/jmsj.2014-302
Pathak R, Sahany S, Mishra SK, Dash SK (2019) Precipitation biases in CMIP5 models over the south Asian region. Sci Rep 9:9589. https://doi.org/10.1038/s41598-019-45907-4
Perkins SE, Pitman AJ, Holbrook NJ, McAneney J (2007) Evaluation of the AR4 climate models’ simulated daily maximum temperature, minimum temperature, and precipitation over Australia using probability density functions. J Clim 20:4356–4376. https://doi.org/10.1175/JCLI4253.1
Raju KS, Kumar DN (2014) Ranking of global climate models for India using multicriterion analysis. Clim Res 60:103–117. https://doi.org/10.3354/cr01222
Raju KS, Kumar DN (2020) Review of approaches for selection and ensembling of GCMS. Journal of Water and Climate Change 11:577–599. https://doi.org/10.2166/wcc.2020.128
Rivera JA, Arnould G (2020) Evaluation of the ability of CMIP6 models to simulate precipitation over Southwestern South America: climatic features and long-term trends (1901–2014). Atmos Res 241:104953. https://doi.org/10.1016/j.atmosres.2020.104953
Sa’adi Z, Shahid S, Chung ES, Bin IT (2017) Projection of spatial and temporal changes of rainfall in Sarawak of Borneo Island using statistical downscaling of CMIP5 models. Atmos Res 197:446–460. https://doi.org/10.1016/j.atmosres.2017.08.002
Salman SA, Shahid S, Ismail T, Ahmed K, Wang XJ (2018) Selection of climate models for projection of spatiotemporal changes in temperature of Iraq with uncertainties. Atmos Res 213:509–522. https://doi.org/10.1016/j.atmosres.2018.07.008
Shashikanth K, Salvi K, Ghosh S, Rajendran K (2014) Do CMIP5 simulations of Indian summer monsoon rainfall differ from those of CMIP3? Atmos Sci Lett 15:79–85. https://doi.org/10.1002/asl2.466
Sheffield J, Barrett AP, Colle B, Nelun Fernando D, Fu R, Geil KL, Hu Q, Kinter J, Kumar S, Langenbrunner B, Lombardo K, Long LN, Maloney E, Mariotti A, Meyerson JE, Mo KC, David Neelin J, Nigam S, Pan Z, Ren T, Ruiz-Barradas A, Serra YL, Seth A, Thibeault JM, Stroeve JC, Yang Z, Yin L (2013) North American climate in CMIP5 experiments. Part I: evaluation of historical simulations of continental and regional climatology. J Clim 26:9209–9245. https://doi.org/10.1175/JCLI-D-12-00592.1
Shiru MS, Shahid S, Dewan A, Chung ES, Alias N, Ahmed K, Hassan QK (2020) Projection of meteorological droughts in Nigeria during growing seasons under climate change scenarios. Sci Rep 10:10107. https://doi.org/10.1038/s41598-020-67146-8
Solomon SD, Qin M, Manning Z, et al (2007) Summary for policymakers. In: Climate Change 2007: The Physical Science Basis. Contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change. D Qin M Manning Z Chen M Marquis K Averyt M Tignor and HL Miller New York Cambridge University Press pp. https://doi.org/10.1038/446727a
Song YH, Nashwan MS, Chung ES, Shahid S (2021) Advances in CMIP6 INM-CM5 over CMIP5 INM-CM4 for precipitation simulation in South Korea. Atmos Res 247:105261. https://doi.org/10.1016/j.atmosres.2020.105261
Sperber KR, Annamalai H, Kang IS, Kitoh A, Moise A, Turner A, Wang B, Zhou T (2013) The Asian summer monsoon: an intercomparison of CMIP5 vs. CMIP3 simulations of the late 20th century. Clim Dyn 41:2711–2744. https://doi.org/10.1007/s00382-012-1607-6
Sun Q, Miao C, Duan Q (2015) Comparative analysis of CMIP3 and CMIP5 global climate models for simulating the daily mean, maximum, and minimum temperatures and daily precipitation over China. J Geophys Res-Atmos 120:4806–4824. https://doi.org/10.1002/2014JD022994
Sun Q, Miao C, Duan Q, Ashouri H, Sorooshian S, Hsu KL (2018) A review of global precipitation data sets: data sources, estimation, and intercomparisons. Rev Geophys 56:79–107. https://doi.org/10.1002/2017RG000574
Taylor KE (2001) Summarizing multiple aspects of model performance in a single diagram. J Geophys Res-Atmos 106:7183–7192. https://doi.org/10.1029/2000JD900719
Taylor KE, Stouffer RJ, Meehl GA (2012) An overview of CMIP5 and the experiment design. Bull Am Meteorol Soc 93:485–498
Tian B, Dong X (2020) The double-ITCZ bias in CMIP3, CMIP5, and CMIP6 models based on annual mean precipitation. Geophys Res Lett 47. https://doi.org/10.1029/2020GL087232
Tiwari PR, Kar SC, Mohanty UC, Kumari S, Sinha P, Nair A, Dey S (2014) Skill of precipitation prediction with GCMs over north India during winter season. Int J Climatol 34:3440–3455. https://doi.org/10.1002/joc.3921
Tokarska KB, Stolpe MB, Sippel S, Fischer EM, Smith CJ, Lehner F, Knutti R (2020) Past warming trend constrains future warming in CMIP6 models. Sci Adv 6:eaaz9549. https://doi.org/10.1126/sciadv.aaz9549
van Vuuren DP, Riahi K (2011) The relationship between short-term emissions and long-term concentration targets. Clim Chang
Wahiduzzaman M, Luo J (2020) A statistical analysis on the contribution of El Niño–Southern Oscillation to the rainfall and temperature over Bangladesh. Meteorog Atmos Phys 1-14:55–68. https://doi.org/10.1007/s00703-020-00733-6
Wu T, Lu Y, Fang Y, Xin X, Li L, Li W, Jie W, Zhang J, Liu Y, Zhang L, Zhang F, Zhang Y, Wu F, Li J, Chu M, Wang Z, Shi X, Liu X, Wei M, Huang A, Zhang Y, Liu X (2019) The Beijing Climate Center Climate System Model (BCC-CSM): the main progress from CMIP5 to CMIP6. Geosci Model Dev 12:1573–1600. https://doi.org/10.5194/gmd-12-1573-2019
Yousuf, A H M (2019) The spectrum of seasonal flooding and drought events in Bangladesh with reference to ENSO and IOD (October 23, 2019). Available at SSRN: https://ssrn.com/abstract=3474176
Zamani Y, Hashemi Monfared SA, Azhdarimoghaddam M, Hamidianpour M (2020) A comparison of CMIP6 and CMIP5 projections for precipitation to observational data: the case of Northeastern Iran. Theor Appl Climatol 142:1613–1623. https://doi.org/10.1007/s00704-020-03406-x
Zambrano-Bigiarini M, Nauditt A, Birkel C, Verbist K, Ribbe L (2017) Temporal and spatial evaluation of satellite-based rainfall estimates across the complex topographical and climatic gradients of Chile. Hydrol Earth Syst Sci 21:1295–1320. https://doi.org/10.5194/hess-21-1295-2017
Zelinka MD, Myers TA, McCoy DT et al (2020) Causes of higher climate sensitivity in CMIP6 models. Geophys Res Lett 47. https://doi.org/10.1029/2019GL085782
Zhou T, Li Z (2002) Simulation of the East Asian summer monsoon using a variable resolution atmospheric GCM. Clim Dyn 19:167–180. https://doi.org/10.1007/s00382-001-0214-8
Zhu YY, Yang S (2020) Evaluation of CMIP6 for historical temperature and precipitation over the Tibetan Plateau and its comparison with CMIP5. Adv Clim Chang Res 11:239–251. https://doi.org/10.1016/j.accre.2020.08.001
Acknowledgements
The authors acknowledge the Bangladesh Meteorological Department (BMD) for providing observational rainfall and temperature data used in this study. Output from the CMIP6 and CMIP5 models from https://esgf-node.llnl.gov/projects/esgf-llnl is greatly acknowledged.
Code availability
The code used in this study is available from the first author upon request (milonbrri@gmail.com).
Author information
Authors and Affiliations
Contributions
Mohammad Kamruzzaman contributes as the main author. Mohammad Kamruzzaman and Shamsuddin Shahid contributed to the study conception, design, material preparation, data collection, and analysis. The first draft of the manuscript was written by Mohammad Kamruzzaman. Shamsuddin Shahid, ARM Towfiqul Islam, Syewoon Hwang, and Jaepil Cho commented on previous versions of the manuscript and reviewed it critically. Md. Asad Uz zaman, Minhaz Ahmed, Md. Mizanur Rahman, and Md. Belal Hossain helped in the preparation of the manuscript and subsequent revisions. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval
Not applicable.
Consent to participate
Not applicable.
Consent for publication
Not applicable.
Conflict of interest
The authors declare no competing interests.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
ESM 1
(DOCX 2245 kb)
Rights and permissions
About this article
Cite this article
Kamruzzaman, M., Shahid, S., Islam, A.T. et al. Comparison of CMIP6 and CMIP5 model performance in simulating historical precipitation and temperature in Bangladesh: a preliminary study. Theor Appl Climatol 145, 1385–1406 (2021). https://doi.org/10.1007/s00704-021-03691-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00704-021-03691-0