Advertisement

Application of Hydrological Forecast Verification Information

  • Kevin WernerEmail author
  • Jan S. VerkadeEmail author
  • Thomas C. PaganoEmail author
Reference work entry

Abstract

Verification studies and systems often focus solely on the exercise of verifying forecasts and not on the application of verification information. This chapter discusses the potential for application of hydrological forecast verification information to improve decision-making in and around the forecast process. Decision-makers include model developers and system designers, forecasters, forecast consumers, and forecast administrators. Each of these has an important role in decisions about forecasts and/or the application of forecasts that may be improved through use of forecast verification. For each, we describe the role, the actions that could be taken to improve forecasts or their application, the context and constraints of those actions, and needs for verification information. Consistent with other studies and assessments on forecast verification, we identify the need for a routine forecast verification system to archive data, plan for operations, measure forecast performance, and group forecasts according to application. Further, we call on forecast agencies and forecast consumers to use forecast verification as a routine part of their operations in order to continually improve services and to engage others to use forecast verification to improve decision-making.

Keywords

Hydrological forecasting Forecast verification Decision making 

References

  1. American Geophysical Union, AGU statement: investigation of scientists and officials in L’Aquila, Italy, is unfounded. EOS Trans. Am. Geophys. Union 91(28), 248 (2010).  https://doi.org/10.1029/2010EO280005CrossRefGoogle Scholar
  2. C. Baldwin, M. Waage, R. Steger, J. Garbrecht, T. Piechota et al., Acclimatizing water managers to climate forecasts through decision experiments, in Climate Variations, Climate Change, and Water Resources Engineering (ASCE, Reston, 2006), pp. 115–131Google Scholar
  3. N.D. Bennett, B.F.W. Croke, G. Guariso, J.H.A. Guillaume, S.H. Hamilton, A.J. Jakeman, S. Marsili-Libelli, L.T.H. Newham, J.P. Norton, C. Perrin, S.A. Pierce, B. Robson, R. Seppelt, A.A. Voinov, B.D. Fath, V. Andreassian, Characterising performance of environmental models. Environ. Model. Softw. 40, 1–20 (2013).  https://doi.org/10.1016/j.envsoft.2012.09.011CrossRefGoogle Scholar
  4. A.A. Bradley, S.S. Schwartz, Summary verification measures and their interpretation for ensemble forecasts. Mon. Weather Rev. 139(9), 3075–3089 (2011).  https://doi.org/10.1175/2010MWR3305.1CrossRefGoogle Scholar
  5. A.A. Bradley, S.S. Schwartz, T. Hashino, Distributions-oriented verification of ensemble streamflow predictions. J. Hydrometeorol. 5(3), 532–545 (2004)CrossRefGoogle Scholar
  6. G.W. Brier, R.A. Allen, Verification of weather forecasts. Compend, in Compendium of Meteorology (1951), pp. 841–848Google Scholar
  7. A.C. Bryant, T.H. Painter, Radiative forcing by desert dust in the Colorado River Basin from 2000 to 2009 inferred from MODIS data, in AGU Fall Meeting Abstracts, vol. 1 (2009), p. 0501. Online Available from http://adsabs.harvard.edu/abs/2009AGUFM.C33B0501B. Accessed 27 Jan 2015
  8. Bureau of Meteorology, Verification in the Bureau. Framework Report (Bureau of Meteorology, Melbourne, 2015)Google Scholar
  9. L. Cuo, T.C. Pagano, Q.J. Wang, A review of quantitative precipitation forecasts and their use in short-to medium-range streamflow forecasting. J. Hydrometeorol. 12(5), 713–728 (2011)CrossRefGoogle Scholar
  10. J. Demargne, M. Mullusky, K. Werner, T. Adams, S. Lindsey, N. Schwein, W. Marosi, E. Welles, Application of forecast verification science to operational river forecasting in the US National Weather Service. Bull. Am. Meteorol. Soc. 90(6), 779–784 (2009)CrossRefGoogle Scholar
  11. Q. Duan, N.K. Ajami, X. Gao, S. Sorooshian, Multi-model ensemble hydrologic prediction using Bayesian model averaging. Adv. Water Resour. 30(5), 1371–1386 (2007)CrossRefGoogle Scholar
  12. B.A. Faber, J.R. Stedinger, Reservoir optimization using sampling SDP with ensemble streamflow prediction (ESP) forecasts. J. Hydrol. 249(1–4), 113–133 (2001).  https://doi.org/10.1016/S0022-1694(01)00419-XCrossRefGoogle Scholar
  13. M.H. Glantz, Consequences and responsibilities in drought forecasting: the case of Yakima, 1977. Water Resour. Res. 18(1), 3–13 (1982)CrossRefGoogle Scholar
  14. N. Gordon, J. Shaykewich, Guidelines on Performance Assessment of Public Weather Services (World Meteorological Organization, Geneva, 2000). Online Available from www.wmo.int/pages/prog/hwrp/documents/FFI/expert/Guidelines_on_Performance_Assessment_of_Public_Weather_Services.pdf
  15. A. Hamlet, D. Huppert, D. Lettenmaier, Economic value of long-lead streamflow forecasts for Columbia River Hydropower. J. Water Resour. Plan. Manag. 128(2), 91–101 (2002).  https://doi.org/10.1061/(ASCE)0733-9496(2002)128:2(91)CrossRefGoogle Scholar
  16. H. Hartmann, R. Bales, S. Sorooshian, Weather, Climate, and Hydrologic Forecasting for the Southwest U.S. (The University of Arizona, Tucson, 1999). Online Available from http://www.climas.arizona.edu/publication/report/weather-climate-and-hydrologic-forecasting-southwest-us
  17. H.C. Hartmann, R. Bales, S. Sorooshian, Weather, climate, and hydrologic forecasting for the US Southwest: a survey. Clim. Res. 21(3), 239–258 (2002)CrossRefGoogle Scholar
  18. I.T. Jolliffe, D.B. Stephenson, Forecast Verification: A Practitioner’s Guide in Atmospheric Science (Wiley, 2012). Online Available from http://books.google.nl/books?hl=en&lr=&id=DCxsKQeaBH8C&oi=fnd&pg=PT8&dq=jolliffe+stephenson&ots=3Ojk_X1AOy&sig=8hGKrwljwaUgKxxYwxzmafhUB8k. Accessed 27 Jan 2015
  19. D. Kahneman, G. Klein, Conditions for intuitive expertise: a failure to disagree. Am. Psychol. 64(6), 515 (2009)CrossRefGoogle Scholar
  20. D. Kavetski, M.P. Clark, Numerical troubles in conceptual hydrology: approximations, absurdities and impact on hypothesis testing. Hydrol. Process. 25(4), 661–670 (2011)CrossRefGoogle Scholar
  21. Y. Liu, A.H. Weerts, M. Clark, H.-J. Hendricks Franssen, S. Kumar, H. Moradkhani, D.-J. Seo, D. Schwanenberg, P. Smith, A. Van Dijk et al., Advancing data assimilation in operational hydrologic forecasting: progresses, challenges, and emerging opportunities. Hydrol. Earth Syst. Sci. 16(10), 3863–3887 (2012)CrossRefGoogle Scholar
  22. B.M. Muir, Trust in automation: Part I. Theoretical issues in the study of trust and human intervention in automated systems. Ergonomics 37(11), 1905–1922 (1994)CrossRefGoogle Scholar
  23. B.M. Muir, N. Moray, Trust in automation. Part II. Experimental studies of trust and human intervention in a process control simulation. Ergonomics 39(3), 429–460 (1996)CrossRefGoogle Scholar
  24. A.H. Murphy, What is a good forecast? An essay on the nature of goodness in weather forecasting. Weather Forecast. 8(2), 281–293 (1993)CrossRefGoogle Scholar
  25. A.H. Murphy, Forecast verification, in Economic Value of Weather and Climate Forecasts (Cambridge University Press, Cambridge, UK/New York/Melbourne, 1997)Google Scholar
  26. N. Nicholls, Cognitive illusions, heuristics, and climate prediction. Bull. Am. Meteorol. Soc. 80(7), 1385–1397 (1999). https://doi.org/10.1175/1520-0477(1999)080<1385:CIHACP>2.0.CO;2CrossRefGoogle Scholar
  27. K. O’Grady, L. Shabman, Communicating the probability of Great Lakes water levels and storms, in Proceedings of Great Lakes Water Level Forecast and Statistics Symposium, Windsor (1990) pp. 197–204Google Scholar
  28. T.C. Pagano, Evaluation of Mekong River commission operational flood forecasts, 2000–2012. Hydrol. Earth Syst. Sci. 18(7), 2645–2656 (2014)CrossRefGoogle Scholar
  29. T.C. Pagano, D. Garen, S. Sorooshian, Evaluation of official western US seasonal water supply outlooks, 1922–2002. J. Hydrometeorol. 5(5), 896–909 (2004)CrossRefGoogle Scholar
  30. T.C. Pagano, H. Hapuarachchi, Q.J. Wang, Continuous Soil Moisture Accounting and Routing Modelling to Support Short Lead-Time Streamflow Forecasting (CSIRO Water for a Healthy Country National Research Flagship, Melbourne, 2009)Google Scholar
  31. T.C. Pagano, A.W. Wood, M.-H. Ramos, H.L. Cloke, F. Pappenberger, M.P. Clark, M. Cranston, D. Kavetski, T. Mathevet, S. Sorooshian, J.S. Verkade, Challenges of operational river forecasting. J. Hydrometeorol. (2014). Online Available from http://journals.ametsoc.org/doi/abs/10.1175/JHM-D-13-0188.1. Accessed 27 Jan 2015
  32. F. Pappenberger, J. Thielen, M. Del Medico, The impact of weather forecast improvements on large scale hydrology: analysing a decade of forecasts of the European Flood Alert System. Hydrol. Process. 25(7), 1091–1113 (2011)CrossRefGoogle Scholar
  33. F. Pappenberger, M.H. Ramos, H.L. Cloke, F. Wetterhall, L. Alfieri, K. Bogner, A. Mueller, P. Salamon, How do I know if my forecasts are better? Using benchmarks in hydrological ensemble prediction. J. Hydrol. 522, 697–713 (2015a).  https://doi.org/10.1016/j.jhydrol.2015.01.024CrossRefGoogle Scholar
  34. F. Pappenberger, H.L. Cloke, D.J. Parker, F. Wetterhall, D.S. Richardson, J. Thielen, The monetary benefit of early flood warnings in Europe. Environ. Sci. Pol. 51, 278–291 (2015b).  https://doi.org/10.1016/j.envsci.2015.04.016CrossRefGoogle Scholar
  35. R. Parasuraman, V. Riley, Humans and automation: use, misuse, disuse, abuse. Hum. Factors J. Hum. Factors Ergon. Soc. 39(2), 230–253 (1997)CrossRefGoogle Scholar
  36. L. Paskus, Why the Silvery Minnow Matters, AlterNet (2003). Online Available from http://www.alternet.org/story/17152/why_the_silvery_minnow_matters. Accessed 27 Jan 2015
  37. C. Perrin, C. Michel, V. Andréassian, Improvement of a parsimonious model for streamflow simulation. J. Hydrol. 279(1), 275–289 (2003)CrossRefGoogle Scholar
  38. R.A. Pielke Jr., Who decides? Forecasts and responsibilities in the 1997 Red River flood. Appl. Behav. Sci. Rev. 7(2), 83–101 (1999)CrossRefGoogle Scholar
  39. M. Pocernich, Appendix: verification software, in Forecast Verification: A Practitioner’s Guide in Atmospheric Science, 2nd edn. (Wiley, Chichester, 2012), pp. 231–240. Online Available from http://onlinelibrary.wiley.com/doi/10.1002/9781119960003.app1/summary. Accessed 27 Jan 2015CrossRefGoogle Scholar
  40. S. Rayner, D. Lach, H. Ingram, Weather forecasts are for wimps: why water resource managers do not use climate forecasts. Clim. Chang. 69(2–3), 197–227 (2005)CrossRefGoogle Scholar
  41. S.L. Rhodes, D. Ely, J.A. Dracup, Climate and the Colorado River: the limits of management. Bull. Am. Meteorol. Soc. 65(7), 682–691 (1984)CrossRefGoogle Scholar
  42. D.E. Robertson, D.L. Shrestha, Q.J. Wang, Post processing rainfall forecasts from numerical weather prediction models for short term streamflow forecasting. Hydrol. Earth Syst. Sci. Discuss. 10(5), 6765–6806 (2013).  https://doi.org/10.5194/hessd-10-6765-2013CrossRefGoogle Scholar
  43. A. Rossa, K. Liechti, M. Zappa, M. Bruen, U. Germann, G. Haase, C. Keil, P. Krahe, The COST 731 action: a review on uncertainty propagation in advanced hydro-meteorological forecast systems. Atmos. Res. 100(2–3), 150–167 (2011).  https://doi.org/10.1016/j.atmosres.2010.11.016CrossRefGoogle Scholar
  44. D. Sarewitz, R.A. Pielke, R. Byerly, Prediction: Science, Decision Making, and the Future of Nature (Island Press, 2000). Online Available from http://books.google.nl/books?hl=en&lr=&id=O0nxEU-deAUC&oi=fnd&pg=PR11&dq=sarewitz+pielke+prediction&ots=F3r_mNYv9p&sig=78GiOAFyglce8xbodoqOVanjRZA. Accessed 27 Jan 2015
  45. L.J. Skitka, K.L. Mosier, M. Burdick, Does automation bias decision-making? Int. J. Hum. Comput. Stud. 51(5), 991–1006 (1999)CrossRefGoogle Scholar
  46. H.R. Stanski, L.J. Wilson, W.R. Burrows, Survey of Common Verification Methods in Meteorology (World Meteorological Organization, Geneva, 1989). Online Available from http://www.eumetcal.org/resources/ukmeteocal/verificationSAV/www/english/msg/library/SWB_Chapter1.pdf. Accessed 27 Jan 2015
  47. J.S. Verkade, M.G.F. Werner, Estimating the benefits of single value and probability forecasting for flood warning. Hydrol. Earth Syst. Sci. 15(12), 3751–3765 (2011).  https://doi.org/10.5194/hess-15-3751-2011CrossRefGoogle Scholar
  48. R.Y. Wang, D.M. Strong, Beyond accuracy: what data quality means to data consumers. J. Manag. Inf. Syst. 12, 5–33 (1996)CrossRefGoogle Scholar
  49. A.H. Weerts, H.C. Winsemius, J.S. Verkade, Estimation of predictive hydrological uncertainty using quantile regression: examples from the National Flood Forecasting System (England and Wales). Hydrol. Earth Syst. Sci. 15(1), 255–265 (2011).  https://doi.org/10.5194/hess-15-255-2011CrossRefGoogle Scholar
  50. E. Welles, Verification of River Stage Forecasts (2005). Online Available from http://arizona.openrepository.com/arizona/handle/10150/195133. Accessed 27 Jan 2015
  51. E. Welles, S. Sorooshian, Scientific verification of deterministic river stage forecasts. J. Hydrometeorol. 10(2), 507–520 (2009)CrossRefGoogle Scholar
  52. E. Welles, S. Sorooshian, G. Carter, B. Olsen, Hydrologic verification: a call for action and collaboration. Bull. Am. Meteorol. Soc. 88(4), 503–511 (2007)CrossRefGoogle Scholar
  53. F. Wetterhall, F. Pappenberger, H.L. Cloke, J. Thielen-del Pozo, S. Balabanova, J. Daňhelka, A. Vogelbacher, P. Salamon, I. Carrasco, A.J. Cabrera-Tordera et al., Forecasters priorities for improving probabilistic flood forecasts. Hydrol. Earth Syst. Sci. Discuss. 10(2), 2215–2242 (2013)CrossRefGoogle Scholar
  54. D.S. Wilks, Statistical Methods in the Atmospheric Sciences (Academic, 2011). Online Available from http://books.google.nl/books?hl=en&lr=&id=IJuCVtQ0ySIC&oi=fnd&pg=PP2&dq=wilks+statistical&ots=anHlqQBLNU&sig=w7ZsTvkiX5BaOjYRzngpdtC9l1M. Accessed 27 Jan2015
  55. World Meteorological Organization, Guide to the Implementation of a Quality Management System for National Meteorological and Hydrological Services (World Meteorological Organization, Geneva, 2013). Online Available from http://www.wmo.int/pages/prog/hwrp/qmf-h/documents/ext/wmo_1100_en.pdf
  56. WWRP/WGNE Joint Working Group on Forecast Verification Research, Forecast Verification: Issues, Method and FAQ (2015). Online Available from http://www.cawcr.gov.au/projects/verification/. Accessed 27 Jan 2015
  57. M. Zappa, K.J. Beven, M. Bruen, A.S. Cofino, K. Kok, E. Martin, P. Nurmi, B. Orfila, E. Roulin, K. Schröter et al., Propagation of uncertainty from observing systems and NWP into hydrological models: COST-731 Working Group 2. Atmos. Sci. Lett. 11(2), 83–91 (2010)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.National Weather Service, National Oceanic and Atmospheric AdministrationSalt Lake CityUSA
  2. 2.DeltaresDelftThe Netherlands
  3. 3.Ministry of Infrastructure and the EnvironmentWater Management Centre of the Netherlands, River Forecasting ServiceLelystadThe Netherlands
  4. 4.Delft University of TechnologyDelftThe Netherlands
  5. 5.Bureau of MeteorologyMelbourneAustralia

Personalised recommendations