Multi-model Combination and Seamless Prediction

  • Stephan HemriEmail author
Reference work entry


(Hydro-) Meteorological predictions are inherently uncertain. Forecasters are trying to estimate and to ultimately also reduce predictive uncertainty. Atmospheric ensemble prediction systems (EPS) provide forecast ensembles that give a first idea of forecast uncertainty. Combining EPS forecasts, issued by different weather services, to multi-model ensembles gives an even better understanding of forecast uncertainty. This article reviews state of the art statistical post-processing methods that allow for sound combinations of multi-model ensemble forecasts. The aim of statistical post-processing is to maximize the sharpness of the predictive distribution subject to calibration. This article focuses on the well-established parametric approaches: Bayesian model averaging (BMA) and ensemble model output statistics (EMOS). Both are readily available and can be used for straightforward implementation of methods for multi-model ensemble combination. Furthermore, methods to ensure seamless predictions in the context of statistical post-processing are summarized. These methods are mainly based on different types of copula approaches. Since skill of (statistically post-processed) ensemble forecasts is generally assessed using particular verification methods, an overview over such methods to verify probabilistic forecasts is provided as well.


Bayesian forecasting system (BFS) Bayesian model averaging Bayesian model averaging (BMA) Box-Cox transformation Brier score (BS) Consortium for Small-Scale Modeling (COSMO) ensemble Development of the European Multi-model Ensemble system for seasonal-to-interannual prediction (DEMETER) DiffeRential Evolution Adaptive Metropolis (DREAM) algorithm Ensemble Kalman filter (EnKF) Ensemble model output statistics (EMOS) Ensemble model output statistics (EMOS) method Ensemble prediction systems (EPS) European Centre for Medium-Range Weather Forecasts (ECMWF) European Flood Alert System (EFAS) Multivariate verification Numerical weather prediction (NWP) model Schaake shuffle Seamless prediction methods Spatiotemporal dependence structures U.S. National Centers for Environmental Prediction (NCEP) Univariate verification Verification 


  1. N. Addor, S. Jaun, F. Fundel, M. Zappa, An operational hydrological ensemble prediction system for the city of Zurich (Switzerland): skill, case studies and scenarios. Hydrol. Earth Syst. Sci. 15, 2327–2347 (2011)CrossRefGoogle Scholar
  2. A. Arribas, K.B. Robertson, K.R. Mylne, Test of a poor man’s ensemble prediction system for short-range probability forecasting. Mon. Weather Rev. 133, 1825–1839 (2005)CrossRefGoogle Scholar
  3. F. Atger, The skill of ensemble prediction systems. Mon. Weather Rev. 127, 1941–1953 (1999)CrossRefGoogle Scholar
  4. J.C. Bartholmes, J. Thielen, M.-H. Ramos, S. Gentilini, The European Flood Alert System EFAS – part 2: statistical skill assessment of probabilistic and deterministic operational forecasts. Hydrol. Earth Syst. Sci. 13, 141–153 (2009)CrossRefGoogle Scholar
  5. Z. Ben Bouallègue, Calibrated short-range ensemble precipitation forecasts using extended logistic regression with interaction terms. Weather Forecast. 28, 515–524 (2013)CrossRefGoogle Scholar
  6. K. Bogner, F. Pappenberger, H.L. Cloke, Technical note: the normal quantile transformation and its application in a flood forecasting system. Hydrol. Earth Syst. Sci. 16, 1085–1094 (2012)CrossRefGoogle Scholar
  7. P. Bougeault, Z. Toth, C. Bishop, B. Brown, D. Burridge, D.H. Chen, B. Ebert, M. Fuentes, T.M. Hamill, K. Mylne, J. Nicolau, T. Paccagnella, Y.-Y. Park, D. Parsons, B. Raoult, D. Schuster, P.S. Dias, R. Swinbank, Y. Takeuchi, W. Tennant, L. Wilson, S. Worley, The THORPEX Interactive Grand Global Ensemble. Bull. Am. Meteorol. Soc. 91, 1059–1072 (2010)CrossRefGoogle Scholar
  8. G. Box, D. Cox, An analysis of transformations. J. R. Stat. Soc. Ser. B Stat. Methodol. 26, 211–252 (1964)Google Scholar
  9. G.W. Brier, Verification of forecasts expressed in terms of probability. Mon. Weather Rev. 78, 1–3 (1950)CrossRefGoogle Scholar
  10. M. Clark, S. Gangopadhyay, L. Rajagalopalan, R. Wilby, The Schaake shuffle: a method for reconstructing space-time variability in forecasted precipitation and temperature fields. J. Hydrometeorol. 5, 243–262 (2004)CrossRefGoogle Scholar
  11. H.L. Cloke, F. Pappenberger, Ensemble flood forecasting: a review. J. Hydrol. 375, 613–626 (2009)CrossRefGoogle Scholar
  12. F.J. Doblas-Reyes, R. Hagedorn, T.N. Palmer, The rationale behind the success of multi-model ensembles in seasonal forecasting – II. Calibration and combination. Tellus A 57, 234–252 (2005)Google Scholar
  13. Q. Duan, N.K. Ajami, X. Gao, S. Sorooshian, Multi-model ensemble hydrologic prediction using Bayesian model averaging. Adv. Water Resour. 30(5), 1371–1386 (2007)CrossRefGoogle Scholar
  14. E.E. Ebert, Ability of a poor man’s ensemble to predict the probability and distribution of precipitation. Mon. Weather Rev. 129, 2461–2480 (2001)CrossRefGoogle Scholar
  15. K. Engeland, I. Steinsland, Probabilistic postprocessing models for flow forecasts for a system of catchments and several lead times. Water Resour. Res. 50, 182–197 (2014)CrossRefGoogle Scholar
  16. European Flood Awareness System, EFAS concepts and tools (2014), Accessed 24 Apr 2015
  17. G. Evensen, Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics. J. Geophys. Res. 99(10), 143–162 (1994)Google Scholar
  18. K. Feldmann, M. Scheuerer, T.L. Thorarinsdottir, Spatial postprocessing of ensemble forecasts for temperature using nonhomogeneous Gaussian regression. Mon. Weather Rev. 143, 955–971 (2015)CrossRefGoogle Scholar
  19. C. Fraley, A.E. Raftery, T. Gneiting, Calibrating multimodel forecast ensembles with exchangeable and missing members using Bayesian model averaging. Mon. Weather Rev. 138(1), 190–202 (2010)CrossRefGoogle Scholar
  20. C. Fraley, A.E. Raftery, J.M. Sloughter, T. Gneiting, University of Washington, ensembleBMA: probabilistic forecasting using ensembles and Bayesian model averaging. R package version 5.1.1 (2015), Accessed 23 Apr 2015
  21. F. Fundel, M. Zappa, Hydrological ensemble forecasting in mesoscale catchments: sensitivity to initial conditions and value of reforecasts. Water Resour. Res. 47, W09520 (2011)CrossRefGoogle Scholar
  22. K.P. Georgakakos, D.-J. Seo, H. Gupta, J. Schaake, M.B. Butts, Towards the characterization of streamflow simulation uncertainty through multimodel ensembles. J. Hydrol. 298, 222–241 (2004)CrossRefGoogle Scholar
  23. H.R. Glahn, D.A. Lowry, The use of model output statistics (MOS) in objective weather forecasting. J. Appl. Meteorol. 11, 1203–1211 (1972)CrossRefGoogle Scholar
  24. T. Gneiting, Calibration of medium-range weather forecasts. ECMWF Technical Memorandum, No. 720 (2014), 28pGoogle Scholar
  25. T. Gneiting, A.E. Raftery, Strictly proper scoring rules, prediction, and estimation. J. Am. Stat. Assoc. 102, 359–378 (2007)CrossRefGoogle Scholar
  26. T. Gneiting, A.E. Raftery, A.H. Westveld, T. Goldman, Calibrated probabilistic forecasting using ensemble model output statistics and minimum CRPS estimation. Mon. Weather Rev. 133(5), 1098–1118 (2005)CrossRefGoogle Scholar
  27. T. Gneiting, F. Balabdoui, A.E. Raftery, Probabilistic forecasts, calibration and sharpness. J. R. Stat. Soc. Ser. B Stat. Methodol. 69, 243–268 (2007)CrossRefGoogle Scholar
  28. T. Gneiting, L.I. Stanberry, E.P. Grimit, L. Held, N.A. Johnson, Assessing probabilistic forecasts of multivariate quantities, with an application to ensemble predictions of surface winds. Test 7(2), 211–235 (2008)CrossRefGoogle Scholar
  29. R. Hagedorn, F.J. Doblas-Reyes, T.N. Palmer, The rationale behind the success of multi-model ensembles in seasonal forecasting – I. Basic concept. Tellus A 57, 219–233 (2005)Google Scholar
  30. R. Hagedorn, R. Buizza, T.M. Hamill, M. Leutbecher, T.N. Palmer, Comparing TIGGE multimodel forecasts with reforecast-calibrated ECMWF ensemble forecasts. Q. J. Roy. Meteorol. Soc. 138(668), 1814–1827 (2012)CrossRefGoogle Scholar
  31. T.M. Hamill, Interpretation of rank histograms for verifying ensemble forecasts. Mon. Weather Rev. 129, 550–560 (2001)CrossRefGoogle Scholar
  32. T.M. Hamill, Verification of TIGGE multimodel and ECMWF reforecast-calibrated probabilistic precipitation forecasts over the contiguous United States. Mon. Weather Rev. 140, 2232–2252 (2012)CrossRefGoogle Scholar
  33. T.M. Hamill, C. Snyder, R.E. Morss, A comparison of probabilistic forecasts from bred, singular-vector, and perturbed observation ensembles. Mon. Weather Rev. 128(6), 1835–1851 (2000)CrossRefGoogle Scholar
  34. S. Hemri, F. Fundel, M. Zappa, Simultaneous calibration of ensemble river flow predictions over an entire range of lead-times. Water Resour. Res. 49 (2013). Scholar
  35. S. Hemri, D. Lisniak, B. Klein, Ascertainment of probabilistic runoff forecasts considering censored data (in German). Hydrol. Wasserbewirtsch. 58(2), 84–94 (2014).,2_4CrossRefGoogle Scholar
  36. S. Hemri, D. Lisniak, B. Klein, Multivariate post-processing techniques for probabilistic hydrological forecasting. Water Resour. Res. 51(9), 7436–7451 (2015)CrossRefGoogle Scholar
  37. H. Hersbach, Decomposition of the continuous ranked probability score for ensemble prediction systems. Weather Forecast. 15(5), 559–570 (2000)CrossRefGoogle Scholar
  38. T. Iversen, A. Deckmyn, C. Santos, K. Sattler, J.B. Bremnes, H. Feddersen, I.-L. Frogner, Evaluation of ‘GLAMEPS’ – a proposed multimodel EPS for short range forecasting. Tellus A 63, 513–530 (2011)CrossRefGoogle Scholar
  39. K. Kober, G.C. Craig, C. Keil, A. Dörnbrack, Blending a probabilistic nowcasting method with a high-resolution numerical weather prediction ensemble for convective precipitation forecasts. Q. J. Roy. Meteorol. Soc. 138, 755–768 (2012)CrossRefGoogle Scholar
  40. K. Kober, G.C. Craig, C. Keil, Aspects of short-term probabilistic blending in different weather regimes. Q. J. Roy. Meteorol. Soc. 140, 1179–1188 (2014)CrossRefGoogle Scholar
  41. T.N. Krishnamurti, C.M. Kishtawal, Z. Zhang, T. LaRow, D. Bachiochi, E. Williford, S. Gadgil, S. Surendran, Improved weather and seasonal climate forecasts from multimodel superensemble. Science 285(5433), 1548–1550 (1999)CrossRefGoogle Scholar
  42. T.N. Krishnamurti, C.M. Kishtawal, Z. Zhang, T. LaRow, D. Bachiochi, E. Williford, S. Gadgil, S. Surendran, Multimodel ensemble forecasts for weather and seasonal climate. J. Climate 13(23), 4196–4216 (2000)CrossRefGoogle Scholar
  43. R. Krzysztofowicz, Bayesian theory of probabilistic forecasting via deterministic hydrologic model. Water Resour. Res. 35(9), 2739–2750 (1999)CrossRefGoogle Scholar
  44. J.E. Matheson, R.L. Winkler, Scoring rules for continuous probability distributions. Manag. Sci. 22, 1087–1096 (1976)CrossRefGoogle Scholar
  45. R. Nelsen, An Introduction to Copulas (Springer, New York, 2006)Google Scholar
  46. T. Palmer, Predicting uncertainty in forecasts of weather and climate. Rep. Prog. Phys. 63, 71–116 (2000)CrossRefGoogle Scholar
  47. T.N. Palmer, F.J. Doblas-Reyes, R. Hagedorn, A. Alessandri, S. Gualdi, U. Andersen, H. Feddersen, P. Cantelaube, J.-M. Terres, M. Davey, R. Graham, P. Délécluse, A. Lazar, M. Déqué, J.-F. Guérémy, E. Díez, B. Orfila, M. Hoshen, A.P. Morse, N. Keenlyside, M. Latif, E. Maisonnave, P. Rogel, V. Marletto, M.C. Thomson, Development of a European multimodel ensemble system for seasonal-to-interannual prediction (DEMETER). Bull. Am. Meteorol. Soc. 85, 853–872 (2004)CrossRefGoogle Scholar
  48. T.N. Palmer, F.J. Doblas-Reyes, A. Weisheimer, M.J. Rodwell, Toward seamless prediction: calibration of climate change projections using seasonal forecasts. Bull. Am. Meteorol. Soc. 89, 459–470 (2008)CrossRefGoogle Scholar
  49. Y.-Y. Park, R. Buizza, M. Leutbecher, TIGGE: preliminary results on comparing and combining ensembles. Q. J. Roy. Meteorol. Soc. 134, 2029–2050 (2008)CrossRefGoogle Scholar
  50. M.A. Parrish, H. Moradkhani, C.M. DeChant, Toward reduction of model uncertainty: integration of Bayesian model averaging and data assimilation. Water Resour. Res. 48, 1–18 (2012)CrossRefGoogle Scholar
  51. P. Pinson, R. Girard, Evaluating the quality of scenarios of short-term wind power generation. Appl. Energy 96, 12–20 (2012)CrossRefGoogle Scholar
  52. A.E. Raftery, T. Gneiting, F. Balabdaoui, M. Polakowski, Using Bayesian model averaging to calibrate forecast ensembles. Mon. Weather Rev. 133(2), 1155–1174 (2005)CrossRefGoogle Scholar
  53. D.S. Richardson, R. Buizza, R. Hagedorn, TIGGE – first final report. WMO/TD-No. 1273 (2005), Accessed 24 Apr 2015.
  54. J. Rings, J.A. Vrugt, G. Schoups, J.A. Husman, H. Vereecken, Bayesian model averaging using particle filtering and Gaussian mixture modeling: theory, concepts, and simulation experiment. Water Resour. Res. 48, W0552 (2012)CrossRefGoogle Scholar
  55. M. Rosenblatt, Remarks on a multivariate transformation. Ann. Math. Stat. 23, 470–472 (1952)CrossRefGoogle Scholar
  56. J. Schaake, J. Pailleux, J. Thielen, R. Arritt, T. Hamill, L. Luo, E. Martin, D. McCollor, F. Pappenberger, Summary of recommendations of the first workshop on postprocessing and downscaling atmospheric forecasts for hydrologic applications held at Météo-France, Toulouse, France, 15–18 June 2009. Atmos. Sci. Lett. 11, 59–63 (2010)CrossRefGoogle Scholar
  57. R. Schefzik, T.L. Thorarinsdottir, T. Gneiting, Uncertainty quantification in complex simulation models using ensemble copula coupling. Stat. Sci. 28, 616–640 (2013)CrossRefGoogle Scholar
  58. M. Scheuerer, L. Büermann, Spatially adaptive post-processing of ensemble forecasts for temperature. J. R. Stat. Soc. Ser. C 63(3), 405–422 (2014)CrossRefGoogle Scholar
  59. K. Scheufele, K. Kober, G.C. Craig, C. Keil, Combining probabilistic precipitation forecasts from a nowcasting technique with a time-lagged ensemble. Meteorol. Appl. 21, 230–240 (2014)CrossRefGoogle Scholar
  60. C. Schölzel, P. Friederichs, Multivariate non-normally distributed random variables in climate research – introduction to the copula approach. Nonlinear Processes Geophys. 15, 761–772 (2008)CrossRefGoogle Scholar
  61. A. Sklar, Fonctions de répartition à n dimensions et leurs marges. Publ. Inst. Stat. Univ. Paris 8, 229–231 (1959)Google Scholar
  62. J. Thielen, J. Schaake, R. Hartman, R. Buizza, Aims, challenges and progress of the Hydrological Ensemble Prediction Experiment (HEPEX) following the third HEPEX workshop held in Stresa 27 to 29 June 2007. Atmos. Sci. Lett. 9, 29–35 (2008)CrossRefGoogle Scholar
  63. J. Thielen, J.C. Bartholmes, M.-H. Ramos, A. de Roo, The European Flood Alert System – part 1: concept and development. Hydrol. Earth Syst. Sci. 13, 125–140 (2009)CrossRefGoogle Scholar
  64. T.L. Thorarinsdottir, M. Scheuerer, C. Heinz, Assessing the calibration of high-dimensional ensemble forecasts using rank histograms. J. Comput. Graph. Stat. 25, 105–122 (2016)CrossRefGoogle Scholar
  65. E. Todini, A model conditional processor to assess predictive uncertainty in flood forecasting. Int. J. River Basin Manag. 6, 123–137 (2008)CrossRefGoogle Scholar
  66. J.A. Vrugt, B.A. Robinson, Treatment of uncertainty using ensemble methods: comparison of sequential data assimilation and Bayesian model averaging. Water Resour. Res. 43, 1–18 (2007)CrossRefGoogle Scholar
  67. J.A. Vrugt, C.G.H. Diks, H.V. Gupta, W. Bouten, J.M. Verstraten, Improved treatment of uncertainty in hydrologic modeling: combining the strengths of global optimization and data assimilation. Water Resour. Res. 41, W01017 (2005). Scholar
  68. J.A. Vrugt, C.J.F. ter Braak, M.P. Clark, J.M. Hyman, B.A. Robinson, Treatment of input uncertainty in hydrologic modeling: doing hydrology backward with Markov chain Monte Carlo simulation. Water Resour. Res. 44, W00B09 (2008). Scholar
  69. J.A. Vrugt, C.J.F. ter Braak, C.G.H. Diks, B.A. Robinson, J.M. Hyman, D. Higdon, Accelerating Markov chain Monte Carlo simulations by differential evolution with self-adaptive randomized subspace sampling. Int. J. Nonlinear Sci. Numer. Simul. 10(3), 271–288 (2009)CrossRefGoogle Scholar
  70. A. Weinheimer, L.A. Smith, K. Judd, A new view of seasonal forecast skill: bounding boxes from the DEMETER ensemble forecasts. Tellus A 57, 265–279 (2005)CrossRefGoogle Scholar
  71. D.S. Wilks, Extending logistic regression to provide full-probability-distribution MOS forecasts. Meteorol. Appl. 16(3), 361–368 (2009). Scholar
  72. D. Wilks, Statistical Methods in the Atmospheric Sciences (Academic Press, Oxford, 2011)Google Scholar
  73. X. Yuan, E. Wood, M. Liang, Developing a seamless hydrologic forecast system: integrating weather and climate prediction. Geophysical Research Abstracts 16(EGU2014-2268) (2014)Google Scholar
  74. R. Yuen, T. Gneiting, T. Thorarinsdottir, C. Fraley, ensembleMOS: ensemble model output statistics. R package version 0.7 (2013), Accessed 23 Apr 2015
  75. M. Zappa, S. Jaun, U. Germann, A. Walser, F. Fundel, Superposition of three sources of uncertainties in operational flood forecasting chains. Atmos. Res. 100, 246–262 (2011)CrossRefGoogle Scholar
  76. C. Ziehmann, Comparison of a single-model EPS with a multi-model ensemble consisting of a few operational models. Tellus A 52, 280–299 (2000)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Computational StatisticsHITS gGmbHHeidelbergGermany

Personalised recommendations