Water Resources Management

, Volume 30, Issue 3, pp 1217–1237 | Cite as

Are Evolutionary Algorithms Effective in Calibrating Different Artificial Neural Network Types for Streamwater Temperature Prediction?

  • Adam P. Piotrowski
  • Maciej J. Napiorkowski
  • Monika Kalinowska
  • Jaroslaw J. Napiorkowski
  • Marzena Osuch
Article

Abstract

Streamwater temperature may be severely affected by the global warming. Different types of models could be used to evaluate the regime of water temperatures in future climatic conditions, including artificial neural networks. As neural networks have no physical background, they require calibration of large number of parameters. This is typically done by gradient-based algorithms, however there is an ongoing debate on usefulness of metaheuristics for this task. In this paper more than ten Swarm Intelligence and Evolutionary Algorithms, including one developed especially for this study, are tested to train four kinds of artificial neural networks (multi-layer perceptron, product-units, adaptive-network-based fuzzy inference systems and wavelet neural networks) for daily water temperature prediction in a natural river located in temperate climate zone. The results are compared with the ones obtained when the classical Levenberg-Marquardt algorithm is used. Finally, the ensemble aggregating approach is tested. Although the research confirms that most metaheuristics do not suite well for training any kind of neural networks, there are exceptions that include the newly proposed heuristic. However, the gain achieved when using even the best metaheuristics is low, comparing to the effort (computational time and complexity of such algorithms). Using ensemble aggregation approach seems to have higher impact on the model performance than seeking for new training methods.

Keywords

Streamwater temperature prediction Temperate climate zone Artificial neural network Differential evolution Particle swarm optimization Genetic algorithm 

Supplementary material

11269_2015_1222_MOESM1_ESM.doc (154 kb)
ESM 1(DOC 154 kb)

References

  1. Adamowski J, Chan HF (2011) A wavelet neural network conjunction model for groundwater level forecasting. J Hydrol 407:28–40CrossRefGoogle Scholar
  2. Adamowski J, Karapataki C (2010) Comparison of multivariate regression and artificial neural networks for peak urban water-demand forecasting: evaluation of different ANN learning algorithms. J Hydrol Eng 15:729–743CrossRefGoogle Scholar
  3. Al-Zahrani MA, Abo-Monasar A (2015) Urban residential water demand prediction based on artificial neural networks and time series models. Water Resour Manag 29(10):3651–3662CrossRefGoogle Scholar
  4. Bullinaria JA, AlYahya K (2014) Artificial bee colony training of neural networks: comparison with back-propagation. Memetic Comput 6(3):171–182CrossRefGoogle Scholar
  5. Bustillo V, Moatar F, Ducharne A, Thiery D, Poirel A (2014) A multimodel comparison for assessing water temperatures under changing climate conditions via the equilibrium temperature concept: case study of the Middle Loire River, France. Hydrol Process 28:1507–1524CrossRefGoogle Scholar
  6. Buyukyildiz M, Tezel G, Yilmaz V (2014) Estimation of the change in lake water level by artificial intelligence methods. Water Resour Manag 28(13):4747–4763CrossRefGoogle Scholar
  7. Caissie D, Kurylyk BL, St-Hilarie A, El-Jabi N, MacQuarrie KTB (2014) Streambed temperature dynamics and corresponding heat fluxes in small streams experiencing seasonal ice cover. J Hydrol 519:1441–1452CrossRefGoogle Scholar
  8. Cao J, Lin Z, Huang GB (2012) Self-adaptive evolutionary extreme learning machine. Neural Process Lett 36:285–305CrossRefGoogle Scholar
  9. Chen WN, Zhang J, Lin Y, Chen N, Zhan ZH, Chung HSH, Li Y, Shi YH (2013) Particle swarm optimization with an aging leader and challengers. IEEE Trans Evol Comput 17(2):241–258CrossRefGoogle Scholar
  10. Chenard JF, Caissie D (2008) Stream temperature modeling using artificial neural networks: application on Catamaran Brook, New Brunswick, Canada. Hydrol Process 22:3361–3372CrossRefGoogle Scholar
  11. Cole JC, Maloney KO, Schmid M, McKenna JE Jr (2014) Developing and testing temperature models for regulated systems: a case study on the Upper Delaware River. J Hydrol 519:588–598CrossRefGoogle Scholar
  12. Daigle A, St-Hilaire A, Ouellet V, Corriveau J, Ouarda TBMJ, Bilodeau L (2009) Diagnostic study and modeling of the annual positive water temperature onset. J Hydrol 370:29–38CrossRefGoogle Scholar
  13. Das S, Abraham A, Chakraborty UK, Konar A (2009) Differential evolution using a neighborhood-based mutation operator. IEEE Trans Evol Comput 13(3):526–553CrossRefGoogle Scholar
  14. Daubechies I (1990) The wavelet transform, time-frequency localization and signal analysis. IEEE Trans Inf Theory 36:961–1005CrossRefGoogle Scholar
  15. DeWeber JT, Wagner T (2014) A regional neural network ensemble for predicting mean daily river water temperature. J Hydrol 517:187–200CrossRefGoogle Scholar
  16. Durbin R, Rumelhart DE (1989) Product units: a computationally powerful and biologically plausible extension to backpropagation networks. Neural Comput 1:133–142CrossRefGoogle Scholar
  17. Eberhart RC, Kennedy J (1995) A new optimizer using particle swarm theory. In: Proceedings of the Sixth International Symposium on Micromachine and Human Science, Nagoya, Japan, pp.39–43Google Scholar
  18. Elsayed SM, Sarker RA, Essam DL (2014) A new genetic algorithm for solving optimization problems. Eng Appl Artif Intel 27:57–69CrossRefGoogle Scholar
  19. European Environment Agency (2012) Climate change, impacts and vulnerability in Europe 2012. An indicator-based report. EEA Report No 12/2012Google Scholar
  20. Gong WY, Cai ZH, Wang Y (2014) Repairing the crossover rate in adaptive differential evolution. Appl Soft Comput 15:149–168CrossRefGoogle Scholar
  21. Guo SM, Yang CC (2015) Enhancing differential evolution utilizing eigenvector-based crossover operator. IEEE Trans Evol Comput 19(1):31–49CrossRefGoogle Scholar
  22. Hadzima-Nyarko M, Rabi A, Sperac M (2014) Implementation of Artificial Neural Networks in modeling the water-air temperature relationship of the River Drava. Water Resour Manag 28:1379–1394CrossRefGoogle Scholar
  23. Hagan MT, Menhaj MB (1994) Training feedforward networks with the Marquardt algorithm. IEEE Trans Neural Netw 5:989–993CrossRefGoogle Scholar
  24. Hansen LK, Salamon P (1990) Neural network ensembles. IEEE Trans Pattern Anal Mach Intell 12:993–1001CrossRefGoogle Scholar
  25. Haykin S (1999) Neural networks, a comprehensive foundation. Macmillan College Publishing Co., New YorkGoogle Scholar
  26. Holland IH (1975) Adaptation in natural and artificial systems. University of Michigan Press, Ann ArborGoogle Scholar
  27. Ilonen J, Kamarainen JK, Lampinen J (2003) Differential evolution training algorithm for feed-forward neural networks. Neural Process Lett 17:93–105CrossRefGoogle Scholar
  28. Jang JSR (1993) ANFIS: adaptive-network-based fuzzy inference systems. IEEE Trans Syst Man Cybern 23:665–685CrossRefGoogle Scholar
  29. Jang JSR, Mizutani E (1996) Levenberg-Marquardt method for ANFIS learning. In: Proceedings of Biennial Conference of the North American Fuzzy Information Processing Society NAFIPS ‘96, Berkeley, CA, pp. 87–91Google Scholar
  30. Jeong DI, Daigle D, St-Hilaire A (2013) Development of a stochastic water temperature model and projection of future water temperature and extreme events in the Ouelle river basin in Québec, Canada. River Res Appl 29:805–821CrossRefGoogle Scholar
  31. Kalinowska MB, Rowinski PM, Kubrak J, Mirosław-Swiatek D (2012) Scenarios of the spread of a waste heat discharge in a river – Vistula River case study. Acta Geophys 60:214–231CrossRefGoogle Scholar
  32. Kumar S, Tiwari MK, Chatterjee C, Mishra A (2015) Reservoir inflow forecasting using ensemble models based on neural networks, wavelet analysis and bootstrap method. Water Resour Manag 29:4863–4883CrossRefGoogle Scholar
  33. Kurylyk BL, MacQuarrie KTB, Voss CI (2014) Climate change impacts on the temperature and magnitude of groundwater discharge from shallow, unconfined aquifers. Water Resour Res 50:3253–3274CrossRefGoogle Scholar
  34. Leerink LR, Giles CL, Horne BG, Jabri MA (1995) Learning with product units. Advances in neural information processing systems, Vol. 7. MIT Press, Cambridge, pp. 537–44Google Scholar
  35. Luo YF, Traore SD, Lyu XW, Wang WG, Wang Y, Xie YG, Jiao XY, Fipps G (2015) Medium range daily reference evapotranspiration forecasting by using ANN and public weather forecasts. Water Resour Manag 29(10):3863–3876CrossRefGoogle Scholar
  36. Maier HR, Jain A, Dandy GC, Sudheer KP (2010) Methods used for the development of neural networks for the prediction of water resource variables in river systems: current status and future directions. Environ Model Softw 25:891–909CrossRefGoogle Scholar
  37. Mandischer M (2002) A comparison of evolution strategies and backpropagation for neural network training. Neurocomputing 42:87–117CrossRefGoogle Scholar
  38. Maric I (2013) Optimization of self-organizing polynomial neural networks. Expert Syst Appl 40:4528–4538CrossRefGoogle Scholar
  39. Martinez-Estudillo A, Martinez-Estudillo F, Hervias-Martinez C, Garcia-Pedrajas N (2006) Evolutionary product unit based neural networks for regression. Neural Netw 19:477–486CrossRefGoogle Scholar
  40. Nourani V, Baghanam AH, Adamowski J, Kisi O (2014) Applications of hybrid wavelet–artificial intelligence models in hydrology: a review. J Hydrol 514:358–377CrossRefGoogle Scholar
  41. Parmar KS, Bhardwaj R (2015) River water prediction modeling using neural networks, fuzzy and wavelet coupled model. Water Resour Manag 29:17–33CrossRefGoogle Scholar
  42. Piotrowski AP (2013) Adaptive memetic differential evolution with global and local neighborhood-based mutation operators. Inf Sci 241:164–194CrossRefGoogle Scholar
  43. Piotrowski AP (2014) Differential evolution algorithms applied to neural network training suffer from stagnation. Appl Soft Comput 21:382–406CrossRefGoogle Scholar
  44. Piotrowski AP, Napiorkowski JJ (2012) Product-units neural networks for catchment runoff forecasting. Adv Water Resour 49:97–113CrossRefGoogle Scholar
  45. Piotrowski AP, Napiorkowski JJ (2013) A comparison of methods to avoid overfitting in neural networks training in the case of catchment runoff modeling. J Hydrol 476:97–111CrossRefGoogle Scholar
  46. Piotrowski AP, Osuch M, Napiorkowski MJ, Rowinski PM, Napiorkowski JJ (2014) Comparing large number of metaheuristics for artificial neural networks training to predict water temperature in a natural river. Comput Geosci 64:136–151CrossRefGoogle Scholar
  47. Piotrowski AP, Napiorkowski MJ, Napiorkowski JJ, Osuch M (2015) Comparing various artificial neural network types for water temperature prediction in rivers. J Hydrol 529:302–315CrossRefGoogle Scholar
  48. Poole GC, Berman CH (2001) An ecological perspective on in-stream temperature: natural heat dynamics and mechanisms of human-caused thermal degradation. Environ Manag 27:787–802CrossRefGoogle Scholar
  49. Prechlet L (1998) Automatic early stopping using cross-validation: quantifying the criteria. Neural Netw 11:761–777CrossRefGoogle Scholar
  50. Rabi A, Hadzima-Nyarko M, Sperac M (2015) Modelling river temperature from air temperature in the River Drava (Croatia). Hydrol Sci J 60(9):1490–1507CrossRefGoogle Scholar
  51. Rayne S, Henderson G, Gill P, Forest K (2008) Riparian forest harvesting effects on maximum water temperatures in wetland-sourced headwater streams from the Nicola River watershed, British Columbia, Canada. Water Resour Manag 22(5):565–578CrossRefGoogle Scholar
  52. Sahay RR, Srivastava A (2014) Predicting monsoon floods in rivers embedding wavelet transform, genetic algorithm and neural network. Water Resour Manag 28(2):301–317CrossRefGoogle Scholar
  53. Sahoo GB, Ray C, De Carlo EH (2006) Use of neural network to predict flash flood and attendant water qualities of a mountainous stream on Oahu, Hawaii. J Hydrol 327:525–538CrossRefGoogle Scholar
  54. Sahoo GB, Schladow SG, Reuter JE (2009) Forecasting stream water temperature using regression analysis, artificial neural network, and chaotic non-linear dynamic models. J Hydrol 378:325–342CrossRefGoogle Scholar
  55. Sarker RA, Elsayed SM, Ray T (2014) Differential evolution with dynamic parameters selection for optimization problems. IEEE Trans Evol Comput 18(5):689–707CrossRefGoogle Scholar
  56. Sexton RS, Gupta JND (2000) Comparative valuation of genetic algorithm and back-propagation for training neural networks. Inf Sci 129:45–59CrossRefGoogle Scholar
  57. Shoaib M, Shamseldin AY, Melville BW (2014) Comparative study of different wavelet based neural network models for rainfall-runoff modeling. J Hydrol 515:47–58CrossRefGoogle Scholar
  58. Storn R, Price KV (1997) Differential evolution – a simple and efficient heuristicfor global optimization over continuous spaces. J Glob Optim 11(4):341–359CrossRefGoogle Scholar
  59. Taormina S, Chau KW (2015) Neural network river forecasting with multi-objective fully informed particle swarm optimization. J Hydroinf 17(1):99–113CrossRefGoogle Scholar
  60. van Vliet MTH, Ludwig F, Zwolsman JJG, Weedon GP, Kabat P (2011) Global river temperatures and sensitivity to atmospheric warming and changes in river flow. Water Resour Res 47, W02544Google Scholar
  61. Wang X, Zhao SG (2013) Differential evolution algorithm with self-adaptive population resizing mechanism. Math Probl Eng, art. no. 419372. doi: 10.1155/2013/419372
  62. Wang H, Sun H, Li C, Rahnamayan S, Pan JS (2013a) Diversity enhanced particle swarm optimization with neighborhood search. Inf Sci 223:119–135CrossRefGoogle Scholar
  63. Wang H, Rahnamayan S, Sun H, Omran MGH (2013b) Gaussian bare-bones differential evolution. IEEE Trans Cybernet 43(2):634–647CrossRefGoogle Scholar
  64. Wang Y, Li HX, Huang T, Li L (2014) Differential evolution based on covariance matrix learning and bimodal distribution parameter setting. Appl Soft Comput 18:232–247CrossRefGoogle Scholar
  65. Wang L, Zeng Y, Chen T (2015) Back propagation neural network with adaptive differential evolution algorithm for time series forecasting. Expert Syst Appl 42:855–863CrossRefGoogle Scholar
  66. Webb BW, Hannah DM, Moore RD, Brown LE, Nobilis F (2008) Recent advances in stream and river temperature research. Hydrol Process 22:902–918CrossRefGoogle Scholar
  67. Wehrly KE, Brenden TO, Wang L (2009) A Comparison of statistical approaches for predicting stream temperatures across heterogeneous landscapes. J Am Water Resour Assoc 45:986–997CrossRefGoogle Scholar
  68. Wilamowski BM, Yu H (2010) Neural network learning without backpropagation. IEEE Trans Neural Netw 21(11):1793–1803CrossRefGoogle Scholar
  69. Yu WJ, Shen M, Chen WN, Zhan ZH, Gong YJ, Lin Y, Liu O, Zhang J (2014) Differential evolution with two-level parameter adaptation. IEEE Trans Cybernet 44(7):1080–1099CrossRefGoogle Scholar
  70. Zhang J, Sanderson AC (2009) JADE: adaptive differential evolution with optional external archive. IEEE Trans Evol Comput 13(5):945–958CrossRefGoogle Scholar
  71. Zhang G, Patuwo BE, Hu MY (1998) Forecasting with artificial neural networks: the state of the art. Int J Forecast 14:35–62CrossRefGoogle Scholar
  72. Zheng J (2009) Predicting software reliability with neural network ensembles. Expert Syst Appl 36:2116–2122CrossRefGoogle Scholar
  73. Zhu W, Tang Y, Fahg JA, Zhang W (2013) Adaptive population tuning scheme for differential evolution. Inf Sci 223:164–191CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2016

Authors and Affiliations

  • Adam P. Piotrowski
    • 1
  • Maciej J. Napiorkowski
    • 2
  • Monika Kalinowska
    • 1
  • Jaroslaw J. Napiorkowski
    • 1
  • Marzena Osuch
    • 1
  1. 1.Institute of GeophysicsPolish Academy of SciencesWarsawPoland
  2. 2.Faculty of Building Services, Hydro and Environmental EngineeringWarsaw University of TechnologyWarsawPoland

Personalised recommendations