Abstract
Research within the field of hydrology often focuses on the statistical problem of comparing stochastic to machine learning (ML) forecasting methods. The performed comparisons are based on case studies, while a study providing large-scale results on the subject is missing. Herein, we compare 11 stochastic and 9 ML methods regarding their multi-step ahead forecasting properties by conducting 12 extensive computational experiments based on simulations. Each of these experiments uses 2000 time series generated by linear stationary stochastic processes. We conduct each simulation experiment twice; the first time using time series of 100 values and the second time using time series of 300 values. Additionally, we conduct a real-world experiment using 405 mean annual river discharge time series of 100 values. We quantify the forecasting performance of the methods using 18 metrics. The results indicate that stochastic and ML methods may produce equally useful forecasts.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abrahart RJ, See LM, Dawson CW (2008) Neural network hydroinformatics: maintaining scientific rigour. In: Abrahart RJ, See LM, Solomatine DP (eds) Practical hydroinformatics. Springer, Berlin, pp 33–47. https://doi.org/10.1007/978-3-540-79881-1_3
Abrahart RJ, Anctil F, Coulibaly P, Dawson CW, Mount NJ, See LM, Shamseldin AY, Solomatine DP, Toth E, Wilby RL (2012) Two decades of anarchy? Emerging themes and outstanding challenges for neural network river forecasting. Prog Phys Geog 36(4):480–513. https://doi.org/10.1177/0309133312444943
Abudu S, Cui C, King JP, Abudukadeer K (2010) Comparison of performance of statistical models in forecasting monthly streamflow of Kizil River, China. Water Sci Eng 3(3):269–281. https://doi.org/10.3882/j.issn.1674-2370.2010.03.003
Ahmed NK, Atiya AF, GayarAn NE, El-Shishiny H (2010) An empirical comparison of machine learning models for time series forecasting. Econom Rev 29(5–6):594–621. https://doi.org/10.1080/07474938.2010.481556
Akaike H (1974) A new look at statistical model identification. IEEE Trans Autom Control 19(6):716–723. https://doi.org/10.1109/TAC.1974.1100705
Allaire JJ, Xie Y, McPherson J, Luraschi J, Ushey K, Atkins A, Wickham H, Cheng J, Chang W (2018) rmarkdown: dynamic documents for R. R package version 1.10. https://CRAN.R-project.org/package=rmarkdown
Alpaydin E (2010) Introduction to machine learning, 2nd edn. MIT Press, Cambridge
Anctil F, Filion M, Tournebize J (2009) A neural network experiment on the simulation of daily nitrate-nitrogen and suspended sediment fluxes from a small agricultural catchment. Ecol Model 220(6):879–887. https://doi.org/10.1016/j.ecolmodel.2008.12.021
Arcuri A, Fraser G (2013) Parameter tuning or default values? An empirical investigation in search-based software engineering. Empir Softw Eng 18(3):594–623. https://doi.org/10.1007/s10664-013-9249-9
Armstrong JS (2001) Evaluating forecasting methods. In: Armstrong JS (ed) Principles of forecasting. International series in operations research & management science, vol 30. Springer, Boston, pp 443–472. https://doi.org/10.1007/978-0-306-47630-3_20
Armstrong JS, Collopy F (1992) Error measures for generalizing about forecasting methods: empirical comparisons. Int J Forecast 8(1):69–80. https://doi.org/10.1016/0169-2070(92)90008-W
Assimakopoulos V, Nikolopoulos K (2000) The theta model: a decomposition approach to forecasting. Int J Forecast 16(4):521–530. https://doi.org/10.1016/S0169-2070(00)00066-2
Atiya AF, El-Shoura SM, Shaheen SI, El-Sherif MS (1999) A comparison between neural-network forecasting techniques-case study: river flow forecasting. IEEE Trans Neural Netw 10(2):402–409. https://doi.org/10.1109/72.750569
Ballini R, Soares S, Andrade MG (2001) Multi-step-ahead monthly streamflow forecasting by a neurofuzzy network model. In: IFSA world congress and 20th NAFIPS international conference, pp 992–997. https://doi.org/10.1109/NAFIPS.2001.944740
Biau G (2012) Analysis of a random forests model. J Mach Learn Res 13(Apr):1063–1095
Biau G, Scornet E (2016) A random forest guided tour. TEST 25(2):197–227. https://doi.org/10.1007/s11749-016-0481-7
Billah B, Hyndman RJ, Koehler AB (2005) Empirical information criteria for time series forecasting model selection. J Stat Comput Simul 75(10):831–840. https://doi.org/10.1080/00949650410001687208
Bontempi G (2013) Machine learning strategies for time series prediction. European Business Intelligence Summer School, Hammamet, Lecture. 2013. https://pdfs.semanticscholar.org/f8ad/a97c142b0a2b1bfe20d8317ef58527ee329a.pdf. Accessed 12 Sept 2018
Box GEP, Jenkins GM (1968) Some recent advances in forecasting and control. J R Stat Soc C Appl 17(2):91–109. https://doi.org/10.2307/2985674
Breiman L (1996) Bagging predictors. Mach Learn 24(2):123–140. https://doi.org/10.1007/BF00058655
Breiman L (2001a) Random forests. Mach Learn 45(1):5–32. https://doi.org/10.1023/A:1010933404324
Breiman L (2001b) Statistical modeling: the two cultures (with comments and a rejoinder by the author). Stat Sci 16(3):199–231
Brown RG (1959) Statistical forecasting for inventory control. McGraw-Hill, New York
Carlson RF, MacCormick AJA, Watts DG (1970) Application of linear random models to four annual streamflow series. Water Resour Res 6(4):1070–1078. https://doi.org/10.1029/WR006i004p01070
Cheng CT, Xie JX, Chau KW, Layeghifard M (2008) A new indirect multi-step-ahead prediction model for a long-term hydrologic prediction. J Hydrol 361(1–2):118–130. https://doi.org/10.1016/j.jhydrol.2008.07.040
Cheng KS, Lien YT, Wu YC, Su YF (2017) On the criteria of model performance evaluation for real-time flood forecasting. Stoch Environ Res Risk Assess 31(5):1123–1146. https://doi.org/10.1007/s00477-016-1322-7
Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20(3):273–297. https://doi.org/10.1007/BF00994018
Cortez P (2010) Data mining with neural networks and support vector machines using the R/rminer tool. In: Perner P (ed) Advances in data mining. Applications and theoretical aspects. Springer, Berlin, pp 572–583. https://doi.org/10.1007/978-3-642-14400-4_44
Cortez P (2016) rminer: data mining classification and regression methods. R package version 1.4.2. https://CRAN.R-project.org/package=rminer
Criss RE, Winston WE (2008) Do Nash values have value? Discussion and alternate proposals. Hydrol Process 22:2723–2725. https://doi.org/10.1002/hyp.7072
De Gooijer JG, Hyndman RJ (2006) 25 Years of time series forecasting. Int J Forecast 22(3):443–473. https://doi.org/10.1016/j.ijforecast.2006.01.001
De Livera AM, Hyndman RJ, Snyder RS (2011) Forecasting time series with complex seasonal patterns using exponential smoothing. J Am Stat Assoc 106(496):1513–1527. https://doi.org/10.1198/jasa.2011.tm09771
De Vos NJ (2013) Echo state networks as an alternative to traditional artificial neural networks in rainfall–runoff modelling. Hydrol Earth Syst Sci 17:253–267. https://doi.org/10.5194/hess-17-253-2013
Fildes R (1992) The evaluation of extrapolative forecasting methods. Int J Forecast 8(1):81–98. https://doi.org/10.1016/0169-2070(92)90009-X
Fraley C, Leisch F, Maechler M, Reisen V, Lemonte A (2012) fracdiff: fractionally differenced ARIMA aka ARFIMA(p,d,q) models. R package version 1.4-2. https://CRAN.R-project.org/package=fracdiff
Gardner ES Jr (1985) Exponential smoothing: the state of the art. J Forecast 4(1):1–28. https://doi.org/10.1002/for.3980040103
Gardner ES Jr (2006) Exponential smoothing: the state of the art—part II. Int J Forecast 22(4):637–666. https://doi.org/10.1016/j.ijforecast.2006.03.005
GRDC (2017) Long-term statistics and annual characteristics of GRDC timeseries data. Online provided by the Global Runoff Data Centre of WMO. Koblenz: Federal Institute of Hydrology (BfG). Date of retrieval 06 Jan 2018. http://www.bafg.de/GRDC/EN/03_dtprdcts/32_LTMM/longtermstat_node.html
Guo J, Zhou J, Qin H, Zou Q, Li Q (2011) Monthly streamflow forecasting based on improved support vector machine model. Expert Syst Appl 38(10):13073–13081. https://doi.org/10.1016/j.eswa.2011.04.114
Gupta HV, Kling H, Yilmaz KK, Martinez GF (2009) Decomposition of the mean squared error and NSE performance criteria: implications for improving hydrological modelling. J Hydrol 377(1–2):80–91. https://doi.org/10.1016/j.jhydrol.2009.08.003
Harvey AC (1984) A unified view of statistical forecasting procedures. J Forecast 3(3):245–275. https://doi.org/10.1002/for.3980030302
Haslett J, Raftery AE (1989) Space-time modelling with long-memory dependence: assessing Ireland’s wind power resource. J R Stat Soc C Appl 38(1):1–50. https://doi.org/10.2307/2347679
Hastie T, Tibshirani R, Friedman JH (2009) The elements of statistical learning: data mining, inference, and prediction, 2nd edn. Springer, New York
He Z, Wen X, Liu H, Du J (2014) A comparative study of artificial neural network, adaptive neuro fuzzy inference system and support vector machine for forecasting river flown in the semiarid mountain region. J Hydrol 509:379–386. https://doi.org/10.1016/j.jhydrol.2013.11.054
Holt CC (2004) Forecasting seasonals and trends by exponentially weighted moving averages. Int J Forecast 20(1):5–10. https://doi.org/10.1016/j.ijforecast.2003.09.015
Hong WC (2008) Rainfall forecasting by technological machine learning models. Appl Math Comput 200(1):41–57. https://doi.org/10.1016/j.amc.2007.10.046
Hong T, Fan S (2016) Probabilistic electric load forecasting: a tutorial review. Int J Forecast 32(3):914–938. https://doi.org/10.1016/j.ijforecast.2015.11.011
Hothorn T, Leisch F, Zeileis A, Hornik K (2005) The design and analysis of benchmark experiments. J Comput Graph Stat 14(3):675–699. https://doi.org/10.1198/106186005X59630
Hu J, Liu J, Liu Y, Gao C (2001) EMD-KNN model for annual average rainfall forecasting. J Hydrol Eng 18(11):1450–1457. https://doi.org/10.1061/(ASCE)HE.1943-5584.0000481
Humphrey GB, Maier HR, Wu W, Mount NJ, Dandy GC, Abrahart RJ, Dawson CW (2017) Improved validation framework and R-package for artificial neural network models. Environ Modell Softw 92:82–106. https://doi.org/10.1016/j.envsoft.2017.01.023
Hurvich CM, Tsai CL (1993) A corrected Akaike information criterion for vector autoregressive model selection. J Time Ser Anal 14(3):271–279. https://doi.org/10.1111/j.1467-9892.1993.tb00144.x
Hutter F, Lücke J, Schmidt-Thieme L (2015) Beyond manual tuning of hyperparameters. KI 29(4):329–337. https://doi.org/10.1007/s13218-015-0381-0
Hyndman RJ, Athanasopoulos G (2018) Forecasting: principles and practice. OTexts, Melbourne, Australia. https://otexts.org/fpp2/. Accessed 12 Sept 2018
Hyndman RJ, Billah B (2003) Unmasking the Theta method. Int J Forecast 19(2):287–290. https://doi.org/10.1016/S0169-2070(01)00143-1
Hyndman RJ, Khandakar Y (2008) Automatic time series forecasting: the forecast package for R. J Stat Softw 27(3):1–22. https://doi.org/10.18637/jss.v027.i03
Hyndman RJ, Koehler AB (2006) Another look at measures of forecast accuracy. Int J Forecast 22(4):679–688. https://doi.org/10.1016/j.ijforecast.2006.03.001
Hyndman RJ, Koehler AB, Snyder RD, Grose S (2002) A state space framework for automatic forecasting using exponential smoothing methods. Int J Forecast 18(3):439–454. https://doi.org/10.1016/S0169-2070(01)00110-8
Hyndman RJ, Koehler AB, Ord JK, Snyder RD (2005) Prediction intervals for exponential smoothing using two new classes of state space models. J Forecast 24(1):17–37. https://doi.org/10.1002/for.938
Hyndman RJ, Koehler AB, Ord JK, Snyder RD (2008) Forecasting with exponential smoothing: the state space approach. Springer, Berlin, pp 3–7. https://doi.org/10.1007/978-3-540-71918-2
Hyndman RJ, Athanasopoulos G, Bergmeir C, Caceres G, Chhay L, O’Hara-Wild M, Petropoulos F, Razbash S, Wang E, Yasmeen F (2018) forecast: forecasting functions for time series and linear models. R package version 8.4. https://cran.r-project.org/web/packages/forecast/index.html
Jain SK, Das A, Srivastava DK (1999) Application of ANN for reservoir inflow prediction and operation. J Water Res Plan Man 125(5):263–271. https://doi.org/10.1061/(ASCE)0733-9496(1999)125:5(263)
Karatzoglou A, Smola A, Hornik K, Zeileis A (2004) kernlab—an S4 package for kernel methods in R. J Stat Softw 11(9):1–20
Karatzoglou A, Smola A, Hornik K (2018) kernlab: Kernel-Based Machine Learning Lab. R package version 0.9-27. https://cran.r-project.org/web/packages/kernlab/index.html
Kashyap RL (1982) Optimal choice of AR and MA parts in autoregressive moving average models. IEEE Trans Pattern Anal 4(2):99–104. https://doi.org/10.1109/TPAMI.1982.4767213
Khan MS, Coulibaly P (2006) Application of support vector machine in lake water level prediction. J Hydrol Eng 11(3):199–205. https://doi.org/10.1061/(ASCE)1084-0699(2006)11:3(199)
Kim TW, Valdés JB (2003) Nonlinear model for drought forecasting based on a conjunction of wavelet transforms and neural networks. J Hydrol Eng 8(6):319–328. https://doi.org/10.1061/(ASCE)1084-0699(2003)8:6(319)
Kişi Ö (2004) River flow modeling using artificial neural networks. J Hydrol Eng 9(1):60–63. https://doi.org/10.1061/(ASCE)1084-0699(2004)9:1(60)
Kişi Ö (2007) Streamflow forecasting using different artificial neural network algorithms. J Hydrol Eng 12(5):532–539. https://doi.org/10.1061/(ASCE)1084-0699(2007)12:5(532)
Kişi Ö, Cimen M (2011) A wavelet-support vector machine conjunction model for monthly streamflow forecasting. J Hydrol 399(1–2):132–140. https://doi.org/10.1016/j.jhydrol.2010.12.041
Kişi Ö, Cimen M (2012) Precipitation forecasting by using wavelet-support vector machine conjunction model. Eng Appl Artif Intell 25(4):783–792. https://doi.org/10.1016/j.engappai.2011.11.003
Kişi Ö, Shiri J, Nikoofar B (2012) Forecasting daily lake levels using artificial intelligence approaches. Comput Geosci 41:169–180. https://doi.org/10.1016/j.cageo.2011.08.027
Kitanidis PK, Bras RL (1980) Real time forecasting with a conceptual hydrologic model: 2. Applications and results. Water Resour Res 16(6):1034–1044. https://doi.org/10.1029/WR016i006p01034
Kohavi R, John GH (1997) Wrappers for feature subset selection. Artif Intell 97(1–2):273–324. https://doi.org/10.1016/S0004-3702(97)00043-X
Koutsoyiannis D (2010) HESS Opinions “A random walk on water”. Hydrol Earth Syst Sci 14:585–601. https://doi.org/10.5194/hess-14-585-2010
Koutsoyiannis D (2011) Hurst–Kolmogorov dynamics and uncertainty. J Am Water Resour Assoc 47(3):481–495. https://doi.org/10.1111/j.1752-1688.2011.00543.x
Koutsoyiannis D, Montanari A (2015) Negligent killing of scientific concepts: the stationarity case. Hydrol Sci J 60(7–8):1174–1183. https://doi.org/10.1080/02626667.2014.959959
Koutsoyiannis D, Yao H, Georgakakos A (2008) Medium-range flow prediction for the Nile: a comparison of stochastic and deterministic methods. Hydrol Sci J 53(1):142–164. https://doi.org/10.1623/hysj.53.1.142
Krause P, Boyle DP, Bäse F (2005) Comparison of different efficiency criteria for hydrological model assessment. Adv Geosci 5:89–97. https://doi.org/10.5194/adgeo-5-89-2005
Krzysztofowicz R (2001) The case for probabilistic forecasting in hydrology. J Hydrol 249(1–4):2–9. https://doi.org/10.1016/S0022-1694(01)00420-6
Kwiatkowski D, Phillips PCB, Schmidt P, Shin Y (1992) Testing the null hypothesis of stationarity against the alternative of a unit root: how sure are we that economic time series have a unit root? J Econom 54(1–3):159–178. https://doi.org/10.1016/0304-4076(92)90104-Y
Lambrakis N, Andreou AS, Polydoropoulos P, Georgopoulos E, Bountis T (2000) Nonlinear analysis and forecasting of a brackish karstic spring. Water Resour Res 36(4):875–884. https://doi.org/10.1029/1999WR900353
Lanc TL (1992) The importance of input variables to a neural network fault-diagnostic system for nuclear power plants. MSc thesis. https://lib.dr.iastate.edu/rtd/208. Accessed 12 Sept 2018
Legates DR, McCabe GJ Jr (1999) Evaluating the use of “goodness-of-fit” measures in hydrologic and hydroclimatic model validation. Water Resour Res 35(1):233–241. https://doi.org/10.1029/1998WR900018
Liaw A (2018) randomForest: Breiman and Cutler’s random forests for classification and regression. R package version 4.6-14. https://CRAN.R-project.org/package=randomForest
Liaw A, Wiener M (2002) Classification and regression by randomForest. R News 2(3):18–22
Lin JY, Cheng CT, Chau KW (2006) Using support vector machines for long-term discharge prediction. Hydrol Sci J 51(4):599–612. https://doi.org/10.1623/hysj.51.4.599
Liong SY, Sivapragasam C (2002) Flood stage forecasting with support vector machines. J Am Water Resour Assoc 38(1):173–186. https://doi.org/10.1111/j.1752-1688.2002.tb01544.x
Lippmann R (1987) An introduction to computing with neural nets. IEEE ASSP Mag 4(2):4–22. https://doi.org/10.1109/MASSP.1987.1165576
Lu K, Wang L (2011) A novel nonlinear combination model based on support vector machine for rainfall prediction. In: Fourth international joint conference on computational sciences and optimization, p 1343. https://doi.org/10.1109/CSO.2011.50
Luo G (2016) A review of automatic selection methods for machine learning algorithms and hyper-parameter values. Netw Model Anal Health Inform Bioinform 5:18. https://doi.org/10.1007/s13721-016-0125-6
Maier HR, Dandy GC (2000) Neural networks for the prediction and forecasting of water resources variables: a review of modelling issues and applications. Environ Modell Softw 15(1):101–124. https://doi.org/10.1016/S1364-8152(99)00007-9
Makridakis S, Hibon M (2000) The M3-competition: results, conclusions and implications. Int J Forecast 16(4):451–476. https://doi.org/10.1016/S0169-2070(00)00057-1
Makridakis S, Hibon M, Lusk E, Belhadjali M (1987) Confidence intervals: an empirical investigation of the series in the M-competition. Int J Forecast 3(3–4):489–508. https://doi.org/10.1016/0169-2070(87)90045-8
Makridakis S, Spiliotis E, Assimakopoulos V (2018) Statistical and machine learning forecasting methods: concerns and ways forward. PLoS ONE 13(3):e0194889. https://doi.org/10.1371/journal.pone.0194889
Marsland S (2011) Machine learning: an algorithmic perspective, 2nd edn. Chapman and Hall, New York
Millard SP (2013) EnvStats: an R package for environmental statistics. Springer, New York
Millard SP (2018) EnvStats: package for environmental statistics, including US EPA guidance. R package version 2.3.1. https://cran.r-project.org/web/packages/EnvStats/index.html
Mishra AK, Desai VR, Singh VP (2007) Drought forecasting using a hybrid stochastic and neural network model. J Hydrol Eng 12(6):626–638. https://doi.org/10.1061/(ASCE)1084-0699(2007)12:6(626)
Moisen GG (2008) Classification and regression trees. In: Jørgensen SE, Fath BD (eds) Encyclopedia of ecology, vol 1. Elsevier. Oxford, UK, pp 582–588
Montanari A, Rosso R, Taqqu MS (1997) Fractionally differenced ARIMA models applied to hydrologic time series: identification, estimation, and simulation. Water Resour Res 33(5):1035–1044. https://doi.org/10.1029/97WR00043
Montanari A, Rosso R, Taqqu MS (2000) A seasonal fractional ARIMA model applied to the Nile River monthly flows at Aswan. Water Resour Res 36(5):1249–1259. https://doi.org/10.1029/2000WR900012
Murphy AM (1993) What is a good forecast? An essay on the nature of goodness in weather forecasting. Weather Forecast 8:281–293. https://doi.org/10.1175/1520-0434(1993)008%3c0281:WIAGFA%3e2.0.CO;2
Murtagh F (1991) Multilayer perceptrons for classification and regression. Neurocomputing 2(5–6):183–197. https://doi.org/10.1016/0925-2312(91)90023-5
Nash JE, Sutcliffe JV (1970) River flow forecasting through conceptual models part I—a discussion of principles. J Hydrol 10(3):282–290. https://doi.org/10.1016/0022-1694(70)90255-6
Pai PF, Hong WC (2007) A recurrent support vector regression model in rainfall forecasting. Hydrol Process 21:819–827. https://doi.org/10.1002/hyp.6323
Papacharalampous GA (2016) Theoretical and empirical comparison of stochastic and machine learning methods for hydrological processes forecasting. MSc thesis. http://www.itia.ntua.gr/en/docinfo/1670/. Accessed 12 Sept 2018
Papacharalampous GA, Tyralis H (2018) Supplementary material for the paper “Comparison of stochastic and machine learning methods for multi-step ahead forecasting of hydrological processes”. figshare. https://doi.org/10.6084/m9.figshare.7092824
Papacharalampous GA, Tyralis H, Koutsoyiannis D (2017a) Comparison between stochastic and machine learning methods for hydrological multi-step ahead forecasting: all forecasts are wrong!, European Geosciences Union General Assembly 2017, Vienna, Geophysical Research Abstracts, vol 19, EGU2017-3068-2. https://doi.org/10.13140/RG.2.2.17205.47848
Papacharalampous GA, Tyralis H, Koutsoyiannis D (2017b) Error evolution in multi-step ahead streamflow forecasting for the operation of hydropower reservoirs. https://doi.org/10.20944/preprints201710.0129.v1 (Preprints 2017100129)
Papacharalampous GA, Tyralis H, Koutsoyiannis D (2017c) Forecasting of geophysical processes using stochastic and machine learning algorithms. Eur Water 59:161–168
Papacharalampous GA, Tyralis H, Koutsoyiannis D (2018a) One-step ahead forecasting of geophysical processes within a purely statistical framework. Geosci Lett 5:12. https://doi.org/10.1186/s40562-018-0111-1
Papacharalampous GA, Tyralis H, Koutsoyiannis D (2018b) Predictability of monthly temperature and precipitation using automatic time series forecasting methods. Acta Geophys 66(4):807–831. https://doi.org/10.1007/s11600-018-0120-7
Pappenberger F, Ramos MH, Cloke HL, Wetterhall F, Alfieri L, Bogner K, Mueller A, Salamon P (2015) How do I know if my forecasts are better? Using benchmarks in hydrological ensemble prediction. J Hydrol 522:697–713. https://doi.org/10.1016/j.jhydrol.2015.01.024
Patel SS, Ramachandran P (2015) A comparison of machine learning techniques for modeling river flow time series: the case of upper Cauvery river basin. Water Resour Manag 29(2):589–602. https://doi.org/10.1007/s11269-014-0705-0
R Core Team (2018) R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. https://www.R-project.org/
Raghavendra NS, Deka PC (2014) Support vector machine applications in the field of hydrology: a review. Appl Soft Comput 19:372–386. https://doi.org/10.1016/j.asoc.2014.02.002
Ramos MH, Mathevet T, Thielen J, Pappenberger F (2010) Communicating uncertainty in hydro-meteorological forecasts: mission impossible? Meteorol Appl 17(2):223–235. https://doi.org/10.1002/met.202
Ramos MH, Van Andel SJ, Pappenberger F (2013) Do probabilistic forecasts lead to better decisions? Hydrol Earth Syst Sci 17:2219–2232. https://doi.org/10.5194/hess-17-2219-2013
Ripley B (2016) nnet: feed-forward neural networks and multinomial log-linear models. R package version 7.3-12. https://cran.r-project.org/web/packages/nnet/index.html
Sapankevych NI, Sankar R (2009) Time series prediction using support vector machines: a survey. IEEE Comput Intell Mag 4(2):24–38. https://doi.org/10.1109/MCI.2009.932254
Schaefli B, Gupta HV (2007) Do Nash values have value? Hydrol Process 21(15):2075–2080. https://doi.org/10.1002/hyp.6825
Schwarz G (1978) Estimating the dimension of a model. Ann Stat 6(2):461–464. https://doi.org/10.1214/15-AOS1321
Scornet E, Biau G, Vert JP (2015) Consistency of random forests. Ann Stat 43(4):1716–1741
Shabri A, Suhartono (2012) Streamflow forecasting using least-squares support vector machines. Hydrol Sci J 57(7):1275–1293. https://doi.org/10.1080/02626667.2012.714468
Shi Z, Han M (2007) Support vector echo-state machine for chaotic time-series prediction. IEEE Trans Neural Netw 18(2):359–372. https://doi.org/10.1109/TNN.2006.885113
Shmueli G (2010) To explain or to predict? Stat Sci 25(3):289–310. https://doi.org/10.1214/10-STS330
Silver D, Huang A, Maddison C, Guez A, Sifre L, van den Driessche G, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M, Dieleman S, Grewe D, Nham J, Kalchbrenner N, Sutskever I, Lillicrap T, Leach M, Kavukcuoglu K, Graepel T, Hassabis D (2016) Mastering the game of Go with deep neural networks and tree search. Nature 529:484–489. https://doi.org/10.1038/nature16961
Sivakumar B (2004) Chaos theory in geophysics: past, present and future. Chaos Solitons Fractals 19(2):441–462. https://doi.org/10.1016/S0960-0779(03)00055-9
Sivapragasam C, Liong SY, Pasha MFK (2001) Rainfall and runoff forecasting with SSA-SVM approach. J Hydroinform 3(3):141–152
Smola AJ, Schölkopf B (2004) A tutorial on support vector regression. Stat Comput 14(3):199–222. https://doi.org/10.1023/B:STCO.0000035301.49549.88
Solomatine DP, Ostfeld A (2008) Data-driven modelling: some past experiences and new approaches. J Hydroinform 10(1):3–22. https://doi.org/10.2166/hydro.2008.015
Sutton CD (2005) Classification and regression trees, bagging, and boosting. Handb Stat 24:303–329. https://doi.org/10.1016/S0169-7161(04)24011-1
Thissen U, Van Brakel R, De Weijer AP, Melssena WJ, Buydens LMC (2003) Using support vector machines for time series prediction. Chemom Intell Lab 69(1–2):35–49. https://doi.org/10.1016/S0169-7439(03)00111-4
Tyralis H (2016) HKprocess: Hurst–Kolmogorov process. R package version 0.0-2. https://CRAN.R-project.org/package=HKprocess
Tyralis H, Koutsoyiannis D (2011) Simultaneous estimation of the parameters of the Hurst–Kolmogorov stochastic process. Stoch Environ Res Risk Assess 25(1):21–33. https://doi.org/10.1007/s00477-010-0408-x
Tyralis H, Koutsoyiannis D (2014) A Bayesian statistical model for deriving the predictive distribution of hydroclimatic variables. Clim Dyn 42(11–12):2867–2883. https://doi.org/10.1007/s00382-013-1804-y
Tyralis H, Koutsoyiannis D (2017) On the prediction of persistent processes using the output of deterministic models. Hydrol Sci J 62(13):2083–2102. https://doi.org/10.1080/02626667.2017.1361535
Tyralis H, Papacharalampous GA (2017) Variable selection in time series forecasting using random forests. Algorithms 10(4):114. https://doi.org/10.3390/a10040114
Valipour M, Banihabib ME, Behbahani SMR (2013) Comparison of the ARMA, ARIMA, and the autoregressive artificial neural network models in forecasting the monthly inflow of Dez dam reservoir. J Hydrol 476(7):433–441. https://doi.org/10.1016/j.jhydrol.2012.11.017
Vapnik VN (1995) The nature of statistical learning theory, 1st edn. Springer, New York. https://doi.org/10.1007/978-1-4757-3264-1
Vapnik VN (1999) An overview of statistical learning theory. IEEE Trans Neural Netw 10(5):988–999. https://doi.org/10.1109/72.788640
Venables WN, Ripley BD (2002) Modern applied statistics with S, 4th edn. Springer, New York. https://doi.org/10.1007/978-0-387-21706-2
Wang WC, Chau KW, Cheng CT, Qiu L (2009) A comparison of performance of several artificial intelligence methods for forecasting monthly discharge time series. J Hydrol 374(3–4):294–306. https://doi.org/10.1016/j.jhydrol.2009.06.019
Warnes GR, Bolker B, Gorjanc G, Grothendieck G, Korosec A, Lumley T, MacQueen D, Magnusson A, Rogers J et al (2017) gdata: various R programming tools for data manipulation. R package version 2.18.0. https://CRAN.R-project.org/package=gdata
Wei WWS (2006) Time series analysis, univariate and multivariate methods, 2nd edn. Addison Wesley, Boston
Weijs SV, Schoups G, Van de Giesen N (2010) Why hydrological predictions should be evaluated using information theory. Hydrol Earth Syst Sci 14:2545–2558. https://doi.org/10.5194/hess-14-2545-2010
Wickham H (2011) The split-apply-combine strategy for data analysis. J Stat Softw 40(1):1–29
Wickham H (2016a) ggplot2. Springer, New York. https://doi.org/10.1007/978-3-319-24277-4
Wickham H (2016b) plyr: tools for splitting, applying and combining data. R package version 1.8.4. https://cran.r-project.org/web/packages/plyr/index.html
Wickham H, Chang W (2018) devtools: tools to make developing R packages easier. R package version 1.13.6. https://CRAN.R-project.org/package=devtools
Wickham H, Henry L (2018) tidyr: easily tidy data with ‘spread()’ and ‘gather()’ Functions. R package version 0.8.1. https://CRAN.R-project.org/package=tidyr
Wickham H, Hester J, Francois R, Jylänki J, Jørgensen M (2017) readr: read rectangular text data. R package version 1.1.1. https://CRAN.R-project.org/package=readr
Wickham H, Chang W, Henry L, Pedersen TL, Takahashi K, Wilke C, Woo K (2018) ggplot2: create elegant data visualisations using the grammar of graphics. R package version 3.0. https://cran.r-project.org/web/packages/ggplot2/index.html
Witten IH, Frank E, Hall MA, Pal CJ (2017) Data mining: practical machine learning tools and techniques, fourth edition. Elsevier Inc. ISBN:978-0-12-804291-5
Witthoft C (2015) cgwtools: miscellaneous tools. R package version 3.0. https://cran.r-project.org/src/contrib/Archive/cgwtools/
Wolpert DH (1996) The lack of a priori distinctions between learning algorithms. Neural Comput 8(7):1341–1390. https://doi.org/10.1162/neco.1996.8.7.1341
Xie Y (2014) knitr: A comprehensive tool for reproducible research in R. In: Stodden V, Leisch F, Peng RD (eds) Implementing reproducible computational research. Chapman and Hall, New York
Xie Y (2015) Dynamic documents with R and knitr, 2nd edn. Chapman and Hall, New York
Xie Y (2018) knitr: a general-purpose package for dynamic report generation in R. R package version 1.20. https://cran.r-project.org/web/packages/knitr/index.html
Yapo PO, Gupta HV, Sorooshian S (1996) Automatic calibration of conceptual rainfall-runoff models: sensitivity to calibration data. J Hydrol 181(1–4):23–48. https://doi.org/10.1016/0022-1694(95)02918-4
Yaseen ZM, Allawi MF, Yousif AA, Jaafar O, Hamzah FM, El-Shafie A (2016) Non-tuned machine learning approach for hydrological time series forecasting. Neural Comput Appl 30(5):1479–1491. https://doi.org/10.1007/s00521-016-2763-0
Ye M, Neuman SP, Meyer PD (2004) Maximum likelihood Bayesian averaging of spatial variability models in unsaturated fractured tuff. Water Resour Res 40(5):W05113. https://doi.org/10.1029/2003WR002557
Ye M, Meyer PD, Neuman SP (2008) On model selection criteria in multimodel analysis. Water Resour Res 44(3):W03428. https://doi.org/10.1029/2008WR006803
Yevjevich VM (1987) Stochastic models in hydrology. Stoch Hydrol Hydraul 1(1):17–36. https://doi.org/10.1007/BF01543907
Yu X, Liong SY (2007) Forecasting of hydrologic time series with ridge regression in feature space. J Hydrol 332(3–4):290–302. https://doi.org/10.1016/j.jhydrol.2006.07.003
Zambrano-Bigiarini M (2014) hydroGOF: goodness-of-fit functions for comparison of simulated and observed hydrological time series. R package version 0.3-8. https://CRAN.R-project.org/package=hydroGOF
Zhang GP (2001) An investigation of neural networks for linear time-series forecasting. Comput Oper Res 28(12):1183–1202. https://doi.org/10.1016/S0305-0548(00)00033-2
Zhang GP, Patuwo BE, Hu MY (1998) Forecasting with artificial neural networks: the state of the art. Int J Forecast 14(1):35–62. https://doi.org/10.1016/S0169-2070(97)00044-7
Acknowledgements
We thank the Associate Editor and two reviewers for their useful suggestions. Part of the Discussion section, in particular the comments on the no free lunch theorem and the use of exogenous variables, has been inspired by the “Energy Forecasting” blog (http://blog.drhongtao.com/).
Author information
Authors and Affiliations
Contributions
HT conceived the idea of comparing stochastic and machine learning methods in hydrological univariate time series forecasting using large datasets. GP designed the experiments, performed the computations and wrote the manuscript under the supervision of HT and DK during her MSc thesis. All authors have discussed the results and edited the manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix: Statistical software and supplementary material
Appendix: Statistical software and supplementary material
The analyses and visualizations have been performed in R Programming Language (R Core Team 2018). We have used the following contributed R packages: cgwtools (Witthoft 2015), devtools (Wickham and Chang 2018), EnvStats (Millard 2013, 2018), forecast (Hyndman and Khandakar 2008; Hyndman et al. 2018), fracdiff (Fraley et al. 2012), gdata (Warnes et al. 2017), ggplot2 (Wickham 2016a; Wickham et al. 2018), HKprocess (Tyralis 2016), kernlab (Karatzoglou et al. 2004, 2018), knitr (Xie 2014, 2015, 2018), nnet (Venables and Ripley 2002; Ripley 2016), plyr (Wickham 2011, 2016b), randomForest (Liaw and Wiener 2002; Liaw 2018), readr (Wickham et al. 2017), rmarkdown (Allaire et al. 2018), rminer (Cortez 2010, 2016) and tidyr (Wickham and Henry 2018).
The supplementary material is available in Papacharalampous and Tyralis (2018). We provide the fully reproducible reports together with their codes. We also provide the reports entitled “Definitions of the stochastic processes’’, “Definitions of the forecast quality metrics’’ and “Selected figures for the qualitative comparison of the forecasting methods’’, which we suggest to be read alongside with Sects. 2.1, 2.4 and 3.1 respectively.
Rights and permissions
About this article
Cite this article
Papacharalampous, G., Tyralis, H. & Koutsoyiannis, D. Comparison of stochastic and machine learning methods for multi-step ahead forecasting of hydrological processes. Stoch Environ Res Risk Assess 33, 481–514 (2019). https://doi.org/10.1007/s00477-018-1638-6
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00477-018-1638-6