Climate Dynamics

, Volume 45, Issue 1–2, pp 309–323 | Cite as

Using joint probability distribution functions to evaluate simulations of precipitation, cloud fraction and insolation in the North America Regional Climate Change Assessment Program (NARCCAP)

  • Huikyo Lee
  • Jinwon Kim
  • Duane E. Waliser
  • Paul C. Loikith
  • Chris A. Mattmann
  • Seth McGinnis


This study evaluates model fidelity in simulating relationships between seasonally averaged precipitation, cloud fraction and surface insolation from the North American Regional Climate Change Assessment Project (NARCCAP) hindcast using observational data from ground stations and satellites. Model fidelity is measured in terms of the temporal correlation coefficients between these three variables and the similarity between the observed and simulated joint probability distribution functions (JPDFs) in 14 subregions over the conterminous United States. Observations exhibit strong negative correlations between precipitation/cloud fraction and surface insolation for all seasons, whereas the relationship between precipitation and cloud fraction varies according to regions and seasons. The skill in capturing these observed relationships varies widely among the NARCCAP regional climate models, especially in the Midwest and Southeast coast regions where observations show weak (or even negative) correlations between precipitation and cloud fraction in winter due to frequent non-precipitating stratiform clouds. Quantitative comparison of univariate and JPDFs indicates that model performance varies markedly between regions as well as seasons. This study also shows that comparison of JPDFs is useful for summarizing the performance of and highlighting problems with some models in simulating cloud fraction and surface insolation. Our quantitative metric may be useful in improving climate models by highlighting shortcomings in the formulations related with the physical processes involved in precipitation, clouds and radiation or other multivariate processes in the climate system.


Regional climate model evaluation NARCCAP Cloud, precipitation and radiation Joint probability distribution 



The contributions by H.K., D.E.W., P.C.L. and C.A.M. to this study were carried out on behalf of the Jet Propulsion Laboratory, California Institute of Technology, under a contract with the National Aeronautics and Space Administration. This research was funded by NASA National Climate Assessment 11-NCA11-0028 and AIST AIST-QRS-12-0002 projects, and the NSF ExArch 1125798 (P.C.L., J.K., H.L., and D.E.W).

Supplementary material

382_2014_2253_MOESM1_ESM.pdf (1.7 mb)
Supplementary material 1 (PDF 1717 kb)


  1. Adler RF, Huffman GJ, Chang A, Ferraro R, Xie PP, Janowiak J, Rudolf B, Schneider U, Curtis S, Bolvin D, Gruber A, Susskind J, Arkin P, Nelkin E (2003) The version-2 global precipitation climatology project (GPCP) monthly precipitation analysis (1979-present). J Hydrometeorol 4(6):1147–1167. doi: 10.1175/1525-7541 CrossRefGoogle Scholar
  2. Alexander LV, Zhang X, Peterson TC, Caesar J, Gleason B, Tank AMGK, Haylock M, Collins D, Trewin B, Rahimzadeh F, Tagipour A, Kumar KR, Revadekar J, Griffiths G, Vincent L, Stephenson DB, Burn J, Aguilar E, Brunet M, Taylor M, New M, Zhai P, Rusticucci M, Vazquez-Aguirre JL (2006) Global observed changes in daily climate extremes of temperature and precipitation. J Geophys Res Atmos 111(D5). doi: 10.1029/2005jd006290
  3. Bishop JKB, Rossow WB (1991) Spatial and temporal variability of global surface solar irradiance. J Geophys Res Oceans 96(C9):16839–16858. doi: 10.1029/91jc01754 CrossRefGoogle Scholar
  4. Changnon SA, Huff FA (1957) Cloud distribution and correlation with precipitation in Illinois, Report of Investigation33, Illinois State Water SurveyGoogle Scholar
  5. Crichton DJ, Mattmann CA, Cinquini L, Braverman A, Waliser D, Gunson M, Hart AF, Goodale CE, Lean P, Kim J (2012) Sharing satellite observations with the climate-modeling community: software and architecture. IEEE Softw 29(5):73–81CrossRefGoogle Scholar
  6. Durman CF, Gregory JM, Hassell DC, Jones RG, Murphy JM (2001) A comparison of extreme European daily precipitation simulated by a global and a regional climate model for present and future climates. Q J R Meteorol Soc 127(573):1005–1015. doi: 10.1256/Smsqj.57315 CrossRefGoogle Scholar
  7. Frei C, Christensen JH, Deque M, Jacob D, Jones RG, Vidale PL (2003) Daily precipitation statistics in regional climate models: evaluation and intercomparison for the European Alps. J Geophys Res Atmos 108(D3). doi: 10.1029/2002jd002287
  8. Fu Q, Liou KN (1993) Parameterization of the radiative properties of cirrus clouds. J Atmos Sci 50(13):2008–2025. doi: 10.1175/1520-0469 CrossRefGoogle Scholar
  9. Giorgi F, Bates GT (1989) The climatological skill of a regional model over complex terrain. Mon Weather Rev 117(11):2325–2347. doi: 10.1175/1520-0493 CrossRefGoogle Scholar
  10. Giorgi F, Brodeur CS, Bates GT (1994) Regional climate-change scenarios over the United-States produced with a nested regional climate model. J Clim 7(3):375–399. doi: 10.1175/1520-0442 CrossRefGoogle Scholar
  11. Giorgi F, Hurrell JW, Marinucci MR, Beniston M (1997) Elevation dependency of the surface climate change signal: a model study. J Clim 10(2):288–296. doi: 10.1175/1520-0442 CrossRefGoogle Scholar
  12. Gupta SK, Ritchey NA, Wilber AC, Whitlock CH, Gibson GG, Stackhouse PW (1999) A climatology of surface radiation budget derived from satellite data. J Clim 12(8):2691–2710. doi: 10.1175/1520-0442 CrossRefGoogle Scholar
  13. Hart AF, Goodale CE, Mattmann CA, Zimdars P, Crichton D, Lean P, Kim J, Walise D (2011) A cloud-enabled regional climate model evaluation system. In: 2nd international workshop on software engineering for cloud computing, SECLOUD’11, Co-located with ICSE 2011, Waikiki, Honolulu, HI, pp 43–49Google Scholar
  14. Higgins RW, Shi W, Yarosh E (2000) Improved United States precipitation quality control system and analysis. NCEP/Climate Prediction Center Atlas 7. National Centers for Environmental PredictionGoogle Scholar
  15. Hinkelman LM, Stackhouse PW, Wielicki BA, Zhang TP, Wilson SR (2009) Surface insolation trends from satellite and ground measurements: comparisons and challenges. J Geophys Res Atmos 114. doi: 10.1029/2008jd011004
  16. Hogg RV, Tanis EA (2010) Probability and statistical inference, 8th edn. Prentice Hall, Upper Saddle RiverGoogle Scholar
  17. Huffman GJ, Adler RF, Bolvin DT, Gu GJ, Nelkin EJ, Bowman KP, Hong Y, Stocker EF, Wolff DB (2007) The TRMM multisatellite precipitation analysis (TMPA): quasi-global, multiyear, combined-sensor precipitation estimates at fine scales. J Hydrometeorol 8(1):38–55. doi: 10.1175/Jhm560.1 CrossRefGoogle Scholar
  18. Kanamitsu M, Ebisuzaki W, Woollen J, Yang SK, Hnilo JJ, Fiorino M, Potter GL (2002) Ncep-Doe Amip-Ii reanalysis (R-2). Bull Am Meteorol Soc 83(11):1631–1643. doi: 10.1175/Bams-83-11-1631 CrossRefGoogle Scholar
  19. Kim J (2002) Precipitation variability associated with the North American Monsoon in the 20th century. Geophys Res Lett 29(13). doi: 10.1029/2001gl014316
  20. Kim J, Lee JE (2003) A multiyear regional climate hindcast for the Western United States using the mesoscale atmospheric simulation model. J Hydrometeorol 4(5):878–890. doi: 10.1175/1525-7541 CrossRefGoogle Scholar
  21. Kim J, Gu Y, Liou KN, Park RJ, Song CK (2012) Direct and semi-direct radiative effects of anthropogenic aerosols in the Western United States: seasonal and geographical variations according to regional climate characteristics. Clim Change 111(3–4):859–877. doi: 10.1007/s10584-011-0169-7 CrossRefGoogle Scholar
  22. Kim J, Waliser DE, Mattmann CA, Mearns LO, Goodale CE, Hart AF, Crichton DJ, McGinnis S, Lee H-K, Loikith PC, Boustani M (2013a) Evaluation of the surface air temperature, precipitation, and insolation over the conterminous U.S. in the NARCCAP multi-RCM hindcast experiment using RCMES. J Clim. doi: 10.1175/JCLI-D-12-00452.1
  23. Kim J et al (2013b) Evaluation of the CORDEX-Africa multi-RCM hindcast: systematic model errors. Clim Dyn. doi: 10.1007/s00382-013-1751-7 Google Scholar
  24. Kim J, Waliser DE, Neiman PJ, Guan B, Ryoo JM, Wick GA (2013c) Effects of atmospheric river landfalls on the cold season precipitation in California. Clim Dyn 40(1–2):465–474. doi: 10.1007/s00382-012-1322-3 CrossRefGoogle Scholar
  25. Kyriakidis PC, Kim J, Miller NL (2001) Geostatistical mapping of precipitation from rain gauge data using atmospheric and terrain characteristics. J Appl Meteorol 40(11):1855–1877. doi: 10.1175/1520-0450 CrossRefGoogle Scholar
  26. Lee H, Youn D, Patten KO, Olsen SC, Wuebbles DJ (2012) Diagnostic tools for evaluating quasi-horizontal transport in global-scale chemistry models. J Geophys Res Atmos 117. doi: 10.1029/2012jd017644
  27. Li LF, Li WH, Kushnir Y (2012) Variation of the North Atlantic subtropical high western ridge and its implication to Southeastern US summer precipitation. Clim Dyn 39(6):1401–1412. doi: 10.1007/S00382-011-1214-Y CrossRefGoogle Scholar
  28. Li JLF, Waliser DE, Stephens G, Lee S, L’Ecuyer T, Kato S, Loeb N, Ma HY (2013) Characterizing and understanding radiation budget biases in CMIP3/CMIP5 GCMs, contemporary GCM, and reanalysis. J Geophys Res Atmos 118(15):8166–8184. doi: 10.1002/Jgrd.50378 CrossRefGoogle Scholar
  29. Liang XZ, Pan JP, Zhu JH, Kunkel KE, Wang JXL, Dai A (2006) Regional climate model downscaling of the U.S. summer climate and future change. J Geophys Res Atmos 111(D10). doi: 10.1029/2005jd006685
  30. Liu Y, Wu W, Jensen MP, Toto T (2011) Relationship between cloud radiative forcing, cloud fraction and cloud albedo, and new surface-based approach for determining cloud albedo. Atmos Chem Phys 11(14):7155–7170. doi: 10.5194/Acp-11-7155-2011 CrossRefGoogle Scholar
  31. Mattmann C, Waliser D, Kim J, Goodale C, Hart A, Ramirez P, Crichton D, Zimdars P, Boustani M, Lee H., Loikith, P, Whitehall K, Jack C, Hewitson B (2013) Cloud computing and virtualization within the regional climate model and evaluation system. Earth Sci Inf 7(1):1–12.
  32. Loikith PC, Lintner BR, Kim J, Lee H, Neelin JD, Waliser DE (2013) Classifying reanalysis surface temperature probability density functions (PDFs) over North America with cluster analysis. Geophys Res Lett 40(14):3710–3714. doi: 10.1002/Grl.50688 CrossRefGoogle Scholar
  33. Matsuura K and C Willmott (2009) Terrestrial air temperature and precipitation: 1900–2008 gridded monthly time series (V2.01). Center for Climatic Research, Department of Geography, University of Delaware.
  34. Mearns LO, Arritt R, Biner S, Bukovsky MS, McGinnis S, Sain S, Caya D, Correia J, Flory D, Gutowski W, Takle ES, Jones R, Leung R, Moufouma-Okia W, McDaniel L, Nunes AMB, Qian Y, Roads J, Sloan L, Snyder M (2012) The North American Regional Climate Change Assessment Program overview of phase I results. Bull Am Meteorol Soc 93(9):1337–1362CrossRefGoogle Scholar
  35. Mitchell TD, Jones PD (2005) An improved method of constructing a database of monthly climate observations and associated high-resolution grids. Int J Climatol 25(6):693–712. doi: 10.1002/Joc.1181 CrossRefGoogle Scholar
  36. Morrison H, Gettleman A (2008) A new two-moment stratiform cloud microphysics scheme in the Community Atmospheric Model version 3 (CAM3). Part I: description and numerical tests. J Clim 21:3642–3659CrossRefGoogle Scholar
  37. Perkins SE, Pitman AJ, Holbrook NJ, McAneney J (2007) Evaluation of the AR4 climate models’ simulated daily maximum temperature, minimum temperature, and precipitation over Australia using probability density functions. J Clim 20(17):4356–4376. doi: 10.1175/Jcli4253.1 CrossRefGoogle Scholar
  38. Richards F, Arkin P (1981) On the relationship between satellite-observed cloud cover and precipitation. Mon Weather Rev 109(5):1081–1093. doi: 10.1175/1520-0493 CrossRefGoogle Scholar
  39. Rossow WB, Duenas EN (2004) The International Satellite Cloud Climatology Project (ISCCP) web site—an online resource for research. Bull Am Meteorol Soc 85(2):167–172. doi: 10.1175/Bams-85-2-167 CrossRefGoogle Scholar
  40. Rummukainen M (2010) State-of-the-art with regional climate models. Wires Clim Change 1(1):82–96. doi: 10.1002/Wcc.008 CrossRefGoogle Scholar
  41. Rutledge SA, Hobbs PV (1983) The mesoscale and microscale structure and organization of clouds and precipitation in mid-latitude cyclones. 8. A model for the seeder–feeder process in warm-frontal rainbands. J Atmos Sci 40(5):1185–1206. doi: 10.1175/1520-0469 CrossRefGoogle Scholar
  42. Silverman BW (1998) Density estimation for statistics and data analysis. Monographs on statistics and applied probability, vol 26. Chapman & Hall/CRC, Boca RatonGoogle Scholar
  43. Solomon S, Intergovernmental Panel on Climate Change., Intergovernmental Panel on Climate Change. Working Group I (2007) Climate change 2007: the physical science basis: contribution of working group I to the fourth assessment report of the intergovernmental panel on climate change. Cambridge University Press, CambridgeGoogle Scholar
  44. Soni VK, Pandithurai G, Pai DS (2012) Evaluation of long-term changes of solar radiation in India. Int J Climatol 32(4):540–551. doi: 10.1002/Joc.2294 CrossRefGoogle Scholar
  45. Sparling LC (2000) Statistical perspectives on stratospheric transport. Rev Geophys 38(3):417–436CrossRefGoogle Scholar
  46. Stackhouse Jr PW, Gupta SK, Cox SJ, Mikovitz JC, Zhang T, Hinkelman LM (2011) The NASA/GEWEX surface radiation budget release 3.0: 24.5-year dataset. GEWEX News 21 No. 1Google Scholar
  47. Stephens GL, Vane DG, Boain RJ, Mace GG, Sassen K, Wang ZE, Illingworth AJ, O’Connor EJ, Rossow WB, Durden SL, Miller SD, Austin RT, Benedetti A, Mitrescu C, Team CS (2002) The cloudsat mission and the a-train—A new dimension of space-based observations of clouds and precipitation. Bull Am Meteorol Soc 83(12):1771–1790. doi: 10.1175/Bams-83-12-1771 CrossRefGoogle Scholar
  48. U.S. Global Change Research Program (2009) Global climate change impacts in the United States: a state of knowledge report. Cambridge University Press, CambridgeGoogle Scholar
  49. Waliser DE, Li JLF, Woods CP, Austin RT, Bacmeister J, Chern J, Del Genio A, Jiang JH, Kuang ZM, Meng H, Minnis P, Platnick S, Rossow WB, Stephens GL, Sun-Mack S, Tao WK, Tompkins AM, Vane DG, Walker C, Wu D (2009) Cloud ice: a climate model challenge with signs and expectations of progress. J Geophys Res Atmos 114. doi: 10.1029/2008jd010015
  50. Waliser DE, Li JLF, L’Ecuyer TS, Chen WT (2011) The impact of precipitating ice and snow on the radiation balance in global climate models. Geophys Res Lett 38. doi: 10.1029/2010gl046478
  51. Wang SY, Gillies RR, Takle ES, Gutowski WJ (2009) Evaluation of precipitation in the intermountain region as simulated by the NARCCAP regional climate models. Geophys Res Lett 36. doi: 10.1029/2009gl037930
  52. Weare BC (2004) A comparison of AMIP II model cloud layer properties with ISCCP D2 estimates. Clim Dyn 22(2–3):281–292. doi: 10.1007/S00382-003-0374-9 CrossRefGoogle Scholar
  53. Wehner MF (2013) Very extreme seasonal precipitation in the NARCCAP ensemble: model performance and projections. Clim Dyn 40(1–2):59–80. doi: 10.1007/s00382-012-1393-1 CrossRefGoogle Scholar
  54. Weller GB, Cooley DS, Sain SR (2012) An investigation of the pineapple express phenomenon via bivariate extreme value theory. Environmetrics 23(5):420–439. doi: 10.1002/Env.2143 CrossRefGoogle Scholar
  55. Weller GB, Cooley D, Sain SR, Bukovsky MS, Mearns LO (2013) Two case studies on NARCCAP precipitation extremes. J Geophys Res Atmos 118(18):10475–10489. doi: 10.1002/Jgrd.50824 CrossRefGoogle Scholar
  56. Whitehall et al (2012) Building model evaluation and decision support capacity for CORDEX. WMO Bull 61(2):29–34Google Scholar
  57. Winker DM, Hunt WH, McGill MJ (2007) Initial performance assessment of CALIOP. Geophys Res Lett 34(19). doi: 10.1029/2007gl030135

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Huikyo Lee
    • 1
  • Jinwon Kim
    • 2
  • Duane E. Waliser
    • 1
    • 2
  • Paul C. Loikith
    • 1
  • Chris A. Mattmann
    • 1
    • 2
  • Seth McGinnis
    • 3
  1. 1.Jet Propulsion LaboratoryCalifornia Institute of TechnologyPasadenaUSA
  2. 2.University of California, Los AngelesLos AngelesUSA
  3. 3.The National Center for Atmospheric ResearchBoulderUSA

Personalised recommendations