Evaluation of probabilistic flow predictions in sewer systems using grey box models and a skill score criterion

  • Fannar Örn Thordarson
  • Anders Breinholt
  • Jan Kloppenborg Møller
  • Peter Steen Mikkelsen
  • Morten Grum
  • Henrik Madsen
Original Paper


In this paper we show how the grey box methodology can be applied to find models that can describe the flow prediction uncertainty in a sewer system where rain data are used as input, and flow measurements are used for calibration and updating model states. Grey box models are composed of a drift term and a diffusion term, respectively accounting for the deterministic and stochastic part of the models. Furthermore, a distinction is made between the process noise and the observation noise. We compare five different model candidates’ predictive performances that solely differ with respect to the diffusion term description up to a 4 h prediction horizon by adopting the prediction performance measures; reliability, sharpness and skill score to pinpoint the preferred model. The prediction performance of a model is reliable if the observed coverage of the prediction intervals corresponds to the nominal coverage of the prediction intervals, i.e. the bias between these coverages should ideally be zero. The sharpness is a measure of the distance between the lower and upper prediction limits, and skill score criterion makes it possible to pinpoint the preferred model by taking into account both reliability and sharpness. In this paper, we illustrate the power of the introduced grey box methodology and the probabilistic performance measures in an urban drainage context.


Grey box modelling Interval prediction Reliability Sharpness Skill score Urban drainage 


  1. Baadsgaard M, Nielsen JN, Spliid H, Madsen H, Preisel M (1997) Estimation in stochastic differential equations with a state dependent diffusion term. SYSID’97—11th IFAC symposium of system identification, IFACGoogle Scholar
  2. Barbera PL, Lanza LG, Stagi L (2002) Tipping bucket mechanical errors and their influence on rainfall statistics and extremes. Water Sci Technol 45(2):1–9Google Scholar
  3. Bertrand-Krajewski JL, Bardin JP, Mourad M, Béranger Y (2003) Accounting for sensor calibration, data validation, measurement and sampling uncertainties in monitoring urban drainage systems. Water Sci Technol 47(2):95–102Google Scholar
  4. Breinholt A, Thordarson FÖ, Møller JK, Mikkelsen PS, Grum M, Madsen H (2011) Grey box modelling of flow in sewer systems with state dependent diffusion. Environmetrics 22(8):946–961Google Scholar
  5. Gneiting T, Raftery AE (2007) Strictly proper scoring rules, prediction and estimation. J Am Stat Assoc 102(477):359–378CrossRefGoogle Scholar
  6. Gneiting T, Balabdaoui F, Raftery AE (2007) Probabilistic forecasts, calibration and sharpness. J R Stat Soc B 69(2):243–268CrossRefGoogle Scholar
  7. Giraldo JM, Leirens S, Díaz-Grenados MA, Rodríguez JP (2010) Nonlinear optimization for improving the operation of sewer systems: the Bogotá Case Study. International Environmental Modelling and Software Society (iEMSs). 2010 International Congress on Environmental Modelling and Software Modelling for Environments Sake, Fifth Biennial Meeting, Ottawa, CanadaGoogle Scholar
  8. Iacus SM (2008) Simulation and Inference for stochastic differential equations—with R examples. Springer series of StatisticsGoogle Scholar
  9. Jacobsen JL, Madsen H, Harremoës P (1997) A stochastic model for two-station hydraulics exhibiting transient impact. Water Sci Technol 36(5):19–26CrossRefGoogle Scholar
  10. Jazwinski AH (2007) Stochastic processes and filtering theory. Dover Publications, Mineola, NYGoogle Scholar
  11. Jonsdottir H, Jacobsen, JL, Madsen H (2001) A grey-box model describing the hydraulics in a creek. Environmetrics 12:347–356CrossRefGoogle Scholar
  12. Jonsdottir H, Madsen H, Palsson OP (2006) Parameter estimation in stochastic rainfall-runoff models. J Hydrol 326(1–4):379–393CrossRefGoogle Scholar
  13. Kloeden P, Platen E (1999) Numerical solutions of stochastic differential equations. SpringerGoogle Scholar
  14. Kristensen NR, Madsen H (2003) Continuous time stochastic modeling—CTSM 2.3—mathematics guide. Technical Unversity of DenmarkGoogle Scholar
  15. Kristensen NR, Madsen H, Jørgensen SB (2004a) Parameter estimation in stochastic grey-box models. Automatica 40:225–237CrossRefGoogle Scholar
  16. Kristensen NR, Madsen H, Jørgensen SB (2004b) A method for systematic improvement of stochastic grey-box models. Comput Chem Eng 28(8):1431–1449CrossRefGoogle Scholar
  17. Limpert E, Stahel WA, Abbt M (2001) Log-normal distributions across the sciences: keys and clues. BioScience 51(5):341–352CrossRefGoogle Scholar
  18. Madsen H (2008) Time series analysis. Chapman & Hall/CRCGoogle Scholar
  19. Mannina G, Freni G, Viviani G, Saegrov S, Hafskjold L (2006) Integrated urban water modelling with uncertainty analysis. Water Sci Technol—WST 54(6–7):379–386CrossRefGoogle Scholar
  20. Møller JK, Nielsen HA, Madsen H (2008) Time-adaptive quantile regression. Comput Stat Data Anal 52:1292–1303CrossRefGoogle Scholar
  21. Møller JK, Madsen H, Carstensen J (2011) Parameter estimation in a simple stochastic differential equation for phytoplankton modelling. Ecol Model 222:1793–1799CrossRefGoogle Scholar
  22. Molini A, Lanza LG, Barbera Pl (2005) The impact of tipping-bucket raingauge measurement errors on design rainfall for urban-scale applications. Hydrol Process 19:1073–1088CrossRefGoogle Scholar
  23. Nielsen HA, Madsen H (2006) Modelling the heat consumption in district heating systems using a grey-box approach. Energy Build 38(1):63–71CrossRefGoogle Scholar
  24. Ocampo-Martinez C, Puig V (2010) Piece-wise linear functions-based model predictive control of large-scale sewage systems. IET Control Theory Appl 4(9):1581–1593CrossRefGoogle Scholar
  25. Øksendal B (2003) Stochastic differential equations—an introduction with applications, 6th edn. SpringerGoogle Scholar
  26. Pedersen L, Jensen NE, Christensen LE, Nielsen HA, Madsen H (2010) Quantification of the spatial variability of rainfall based on a dense network of rain gauges. Atmospheric Res 95(4):441–454CrossRefGoogle Scholar
  27. Pinson P, Nielsen HA, Møller JK, Madsen H (2007) Non-parametric probabilistic forecasts of wind power: required properties and evaluation. Wind Energy 10(6):497–516CrossRefGoogle Scholar
  28. Puig V, Cembrano G, Romera J, Quevedo J, Aznar B, Ramoń G, Cabot J (2009) Predictive optimal control of sewer networks using CORAL tool: application to Riera Blanca catchment in Barcelona. Water Sci Technol 60(4):869–878CrossRefGoogle Scholar
  29. Shedekar VS, King KW, Brown LC, Fausey NR, Heckel M, Harmel DR (2009) Measurement errors in tipping bucket rain gauges under different rainfall Intensities and their implication to hydrologic models. Conf. paper, ASABE Annual International Meeting, June 21–24, pp 1–9Google Scholar
  30. Tornøe CW, Jacobsen J, Pedersen O, Hansen T, Madsen H (2006) Grey-box Modelling of pharmacokinetic/pharmacodynamic systems. J Pharmacokinet Pharmacodyn 31(5):401–417CrossRefGoogle Scholar
  31. Vaes G, Willems P, Berlamont J (2005) Areal rainfall correction coefficients for small urban catchments. Atmospheric Res 77(1–4):48–59CrossRefGoogle Scholar
  32. Vestergaard M (1998) Nonlinear filtering in stochastic volatility models. Master’s thesis, Technical University of Denmark. Department of Mathematical Modelling, Lyngby, DenmarkGoogle Scholar
  33. Willems P (2001) Stochastic description of the rainfall input errors in lumped hydrological models. Stoch Environ Res Risk Assess 15:132–152CrossRefGoogle Scholar
  34. Willems P (2010) Parsimonious model for combined sewer overflow pollution. J Environ Eng 136(3):316–325CrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2012

Authors and Affiliations

  • Fannar Örn Thordarson
    • 1
  • Anders Breinholt
    • 2
  • Jan Kloppenborg Møller
    • 1
  • Peter Steen Mikkelsen
    • 2
  • Morten Grum
    • 3
  • Henrik Madsen
    • 1
  1. 1.DTU InformaticsKgs. LyngbyDenmark
  2. 2.DTU EnvironmentKgs. LyngbyDenmark
  3. 3.Krüger, Veolia Water Solutions and TechnologiesSøborgDenmark

Personalised recommendations