Skip to main content
SpringerLink
Log in

An assessment of multi-layer perceptron networks for streamflow forecasting in large-scale interconnected hydrosystems

International Journal of Environmental Science and Technology volume 19, pages 5819–5838 (2022)Cite this article

Abstract

This work analyzes the use of artificial neural networks in the short-term streamflow forecasting for large interconnected hydropower systems. The state-of-the-art optimization algorithms, activation functions, and weight initialization techniques are investigated together with classic methods. We present an algorithm to define the neural network inputs in large hydrosystems and apply it to create models for 55 major hydro plants located in the Paraná Basin, which contribute to more than 30% of the total power generated in Brazil. The paper also compares the performance of the neural networks with the hydrological models that are currently used by the independent system operator to define the dispatch of the electric power generators. Our results show that, overall, the neural network models provide more accurate forecasts than the hydrological models used by the Brazilian System Operator. Finally, the paper discusses the contributions of historical rainfall information in the forecasting of streamflow while using neural network models.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Availability of data and material

The data used in this research are publicly available on the internet.

Code availability

Available upon request.

Abbreviations

Adam:

Adaptive moment estimation

ANN:

Artificial neural network

AR:

Autoregressive

ARMA:

Autoregressive moving average

CPU:

Central processing unit

GD:

Gradient descent

GDM:

Gradient descent with momentum

GPU:

Graphics processing unit

ISO:

Independent system operator

IVS:

Input variable selection

MAPE:

Mean absolute percentage error

MGB-IPH:

Large-scale hydrological model

MLP:

Multi-layer perceptron

MSE:

Mean square error

NSE:

Nash–Sutcliffe efficiency

OWI:

Optimal weight initialization

PAR:

Periodic autoregressive

PARMA:

Periodic autoregressive moving average

ReLU:

Rectified linear unit

RMSprop:

Root-mean-square propagation

SMAP:

Soil moisture accounting procedure

SVM:

Support vector machine

VELMA:

Visualizing ecosystem land management assessments

WI:

Weight initialization

\({\text{In}}\) :

Set of input candidates

\({\text{In}}_{{\text{S}}}^{*}\) :

Best set of streamflow inputs used in a specific ANN

\({\text{Is}}_{{t_{{\text{I}}} }}\) :

Input streamflow candidate (\(s_{{\text{I}}}\), \(t_{{\text{I}}}\))

\({\text{Ip}}_{{t_{{\text{I}}} }}^{{D_{{{\text{AR}}}} }}\) :

Input rainfall candidate accumulated through \(D_{{{\text{AR}}}}\) days (\(p_{{\text{I}}}\), \(t_{{\text{I}}}\)); \(D_{{{\text{AR}}}} = 1\): nonaccumulated rainfall

\({\text{Os}}_{{t_{{\text{o}}} }}\) :

Output streamflow (\(s_{{\text{O}}}\), \(t_{{\text{O}}}\))

\(p_{{\text{I}}} \in {\text{In}}\_ p\) :

Set of rainfall stations considered for input candidates

\(s_{{\text{I}}} \in {\text{In}}\_s\) :

Set of streamflow stations considered for input candidates

\(s_{{\text{O}}}\) :

Output streamflow station

\(t_{{\text{I}}}\) :

Number of days preceding the first day of forecast

\(t_{{\text{O}}}\) :

Day of forecast

\(C\) :

Component of the cost function that penalizes the difference between \(y^{\left( i \right)}\) and \(\hat{y}^{\left( i \right)}\)

\({\text{Cr}}\left( {A,B} \right)\) :

Function that computes the correlation coefficient between the variables \(A\) and \(B\)

\(E_{{{\text{dev}}}} \left( k \right)\) :

Error in the development set until the \(k{\text{th}}\) epoch simulation

\(J\) :

Cost function with \(\ell_{2}\) regularization

\(M2\) :

Cost function (28)

\(R2\) :

Cost function (29)

\(U\left( {a_{1} ,a_{2} } \right)\) :

Continuous uniform distribution in the interval [\(a_{1}\), \(a_{2}\)]

\(U_{{\text{D}}} \left( {a_{1} ,a_{2} } \right)\) :

Discrete uniform distribution in the interval [\(a_{1}\), \(a_{2}\)]

\({\text{Var}}\left[ {w^{\ell } } \right]\) :

Variance of the weights in the \(\ell {\text{th}}\) layer

\(w_{2}^{2}\) :

Square of the \(\ell_{2}\) norm of the weight matrixes

\(\sigma \left( x \right)\) :

Logistic sigmoid activation function

\(b\) :

Bias vector

\(C_{{\text{IS/OS}}}\) :

Lowest correlation coefficient between the streamflow input candidates and output streamflow

\(C_{{{\text{IR}}}}\) :

Maximum correlation coefficient for the input rainfall candidates

\(C_{{\text{IR/OS}}}\) :

Lowest correlation coefficient between the rainfall input candidates and output streamflow

\(C_{{{\text{IS}}}}\) :

Maximum correlation coefficient for the input streamflow candidates

\(db\) :

Gradients of the bias vector

\(dw\) :

Gradients of the weights

\(D_{{{\text{AR}}}}\) :

Number of days lag that rainfall is accumulated

\(D_{{{\text{IS}}}}\) :

Number of days lag of input streamflow investigated

\(D_{{{\text{OS}}}}\) :

Number of days of output streamflow

\(D_{{{\text{IR}}}}\) :

Number of days lag of input rainfall investigated

\(\overline{{{\text{Epoch}}}}\) :

Maximum number of epochs

\(k_{{{\text{prob}}}}\) :

Regularization hyperparameter in dropout

\(m\) :

Number of output variables

\(n\) :

Number of output examples

\(n_{\ell }\) :

Number of neurons in the \(\ell {\text{th}}\) layer

\(N_{{{\text{Hyp}}}}\) :

Number of times that different hyperparameters are tested

\(N_{{{\text{RC}}}}\) :

Number of times that different \(C_{{\text{IR/OS}}}\) and \(C_{{{\text{IR}}}}\) correlation coefficients are tested

\(N_{{{\text{SC}}}}\) :

Number of times that different \(C_{{\text{IS/OS}}}\) and \(C_{{{\text{IS}}}}\) correlation coefficients are tested

\(w\) :

Weights

\(\overline{{y^{\left( i \right)} }}\) :

Average streamflow for the \(i{\text{th}}\) example

\(y^{{\left( {i, t} \right)}}\) :

Estimate for the \(t{\text{th}}\) output variable (day of forecast) in the \(i{\text{th}}\) example

\(\hat{y}^{{\left( {i,t} \right)}}\) :

Expected value for the \(t{\text{th}}\) output variable (day of forecast) in the \(i{\text{th}}\) example

\(\alpha\) :

Learning rate variable

\(\lambda\) :

\(\ell_{2}\) Regularization hyperparameter

\(\mu_{0}\) :

Mean of the output variables \(\hat{y}\)

\(\sigma_{0}\) :

Standard deviation of the output variables \(\hat{y}\)

References

  • ABRACEEL—Brazilian Association of The Energy Traders (2018) Annual Report on the Abraceel Activities (in portuguese)

  • Aggarwal CC (2018) Neural networks and deep learning: a textbook. Springer, Berlin

    Book  Google Scholar 

  • Akhtar MK, Corzo GA, Andel SJ, Jonoski A (2009) River flow forecasting with artificial neural networks using satellite observed precipitation pre-processed with flow length and travel time information: case study of the Ganges river basin. Hydrol Earth Syst Sci 13:1607–1618

    Article  Google Scholar 

  • ANA—Brazilian National Water Agency (2019) Operational data from the reservoirs of the Brazilian electric system, viewed 10 January 2021. www.ana.gov.br (in portuguese)

  • Aytek A, Asce M, Alp M (2008) An application of artificial intelligence for rainfall-runoff modeling. J Earth Syst Sci 117:145–155

    Article  Google Scholar 

  • Bengio Y (2012) Practical recommendations for gradient-based training of deep architectures. Neural networks: tricks of the trade. Springer, Berlin, pp 437–478

    Book  Google Scholar 

  • Box GEP, Jenkins GM, Reinsel GC, Ljung GM (2015) Time series analysis forecasting and control, 5th edn. Wiley, London

    Google Scholar 

  • Bravo JM, Paz AR, Collischonn W, Uvo CB, Pedrollo OC, Chou SC (2009) Incorporating forecasts of rainfall in two hydrologic models used for medium-range streamflow forecasting. J Hydrol Eng 14(5):435–445

    Article  Google Scholar 

  • Chen L, Singh VP, Guo S, Zhou J, Ye L (2014) Copula entropy coupled with artificial neural network for rainfall–runoff simulation. Stoch Environ Res Risk Assess 28:1755–1767

    Article  Google Scholar 

  • Clevert D, Unterthiner T, Hochreiter S (2015) Fast and accurate deep network learning by exponential linear units (ELUs). CoRR abs/1511.0

  • CPTEC—Center for weather forecasting and climate studies of the Brazilian National Institute for Space Research (2019) MERGE, viewed 10 January 2021. http://ftp.cptec.inpe.br/modelos/io/produtos/MERGE/

  • de Queiroz AR, Oliveira FA, Lima JWM, Balestrassi PP (2007) Simulating electricity spot prices in brazil using neural network and design of experiments. IEEE Lausanne Power Tech

  • de Queiroz AR, Lima LMM, Lima JWM, da Silva BC, Scianni LA (2016) Climate change impacts in the energy supply of the Brazilian hydro-dominant power system. Renew Energy 99:379–389

    Article  Google Scholar 

  • Dozat T (2016) Incorporating nesterov momentum into adam. In: International conference on learning representations (ICLR)

  • ECMWF—European Center for Medium-Range Weather Forecasts (2019) viewed 10 January 2021. www.ecmwf.int

  • Egawa T, Suzuki K, Ichikawa Y, Lizaka T, Matsui T, Shikagawa Y (2011) A water flow forecasting for dam using neural networks and regression models. IEEE Power and Energy Society General Meeting.

  • Eslamian S (2014) Handbook of Engineering hydrology: modeling, climate change, and variability. CRC Press, London

    Book  Google Scholar 

  • Faria VAD, de Queiroz AR, Lima LMM, Lima JWM (2018) Cooperative game theory and last addition method in the allocation of firm energy rights. Appl Energy 226:905–915

    Article  Google Scholar 

  • Galelli S, Humphrey GB, Maier HR, Castelletti A, Dandy GC, Gibbs MS (2014) An evaluation framework for input variable selection algorithms for environmental data-driven models. Environ Model Softw 62:33–51

    Article  Google Scholar 

  • Géron A (2017) Hands-on machine learning with Scikit-Learn and TensorFlow. O’ Reily Media

  • Glorot X, Bengio Y (2010) Understanding the difficulty of training deep feedforward neural networks. Proc AISTATS 9:249–256

    Google Scholar 

  • Glorot X, Bordes A, Benigo Y (2011) Deep sparse rectifier neural networks. AISTATS 15:315–323

    Google Scholar 

  • Glorot X, Bordes A, Benigo Y (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15:1929–1958

    Google Scholar 

  • Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press, New York

    Google Scholar 

  • He K, Zhang X, Ren S, Sun J (2015) Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification. ICCV

  • Hinton GE, Deng L, Yu D, Dahl G, Mohamed A, Jaitly N, Senior A, Vanhoucke V, Nguyen P, Sainath T, Kingsbury B (2012a) Deep neural networks for acoustic modeling in speech recognition. IEEE Signal Process Mag 29:82–97

    Article  Google Scholar 

  • Hinton GE, Srivastava N, and Swersky K (2012b) Lecture 6a overview of mini–batch gradient descent. Coursera Lecture slides, viewed 10 January 2021. https://www.cs.toronto.edu/~tijmen/csc321/slides/lecture_slides_lec6.pdf

  • Hinton GE, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov RR (2012c) Improving neural networks by preventing co-adaptation of feature detectors. arXiv:1207.0580v1

  • Hsu K, Gupta HV, Sorooshian S (1995) Artificial neural network modeling of the rainfall-runoff process. Water Resour Res 31:2517–2530

    Article  Google Scholar 

  • Iha—International Hydropower Association (2018) Hydropower status report

  • Jo T, Hou J, Eickholt J, Cheng J (2015) Improving protein fold recognition by deep learning networks. Sci Rep 5:1–11

    Google Scholar 

  • Kingma DP, Ba JL (2015) Adam: a method for stochastic optimization. In: International conference on learning representations (ICLR)

  • KİŞİ Ö (2007) Streamflow forecasting using different artificial neural network algorithms. J Hydrol Eng 12(5):532–539

    Article  Google Scholar 

  • Krizhevsky A, Sutskever I, Hinton G (2012) ImageNet classification with deep convolutional neural networks. In: NIPS

  • Kumar DN, Raju KS, Sathish T (2004) River flow forecasting using recurrent neural networks. Water Resour Manag 18:143–161

    Article  Google Scholar 

  • LeCun Y, Bottou L, Orr GB, Muller KR (1998) Efficient backprop. In: Neural networks, tricks of the trade, Lecture Notes in Computer Science LNCS 1524

  • Legates DR, McCabe JG (1999) Evaluating the use of goodness-of-fit measurements in hydrologic and hydroclimatic model validation. Water Resour Res 35:233–241

    Article  Google Scholar 

  • Li X, Shang W, Wang S (2018) Text-based crude oil price forecasting: a deep learning approach. Int J Forecast 35(5):1548–1560

    Google Scholar 

  • Lima MMM, Popova E, Damien P (2014) Modeling and forecasting of Brazilian reservoir inflows via dynamic linear models. Int J Forecast 30:464–476

    Article  Google Scholar 

  • Lopes I, Montenegro AAA (2017) Hydrological process simulation at plot scale using the smap model in the semiarid 3:78–86

  • Lopes JEG, Braga BPF, Conejo JGL (1982) SMAP—a simplified hydrological model. In: Applied modeling in catchment hydrology Water Resources Publications, Littleton, pp 167–176

  • Maas AL, Hannun AY, Ng AY (2013) Rectifier nonlinearities improve neural network acoustic models. In: International conference on machine learning (ICML)

  • May R, Dandy G, Maier H (2011) Review of input variable selection methods for artificial neural networks artificial neural networks—methodological advances and biomedical applications. InTech

  • Mishra SK, Singh VP (2003) Soil conservation service curve number (SCS-CN) methodology. Springer, Netherlands

    Book  Google Scholar 

  • Moriasi DN, Arnold JG, Van MWL, Bingner RL, Harmel RD, Veith TL (2007) Model evaluation guidelines for systematic quantification of accuracy in watershed simulations. Trans Am Soc Agric Biol Eng 50:885–900

    Google Scholar 

  • Morse G, Stanley KO (2016) Simple evolution optimization can rival stochastic gradient descent in neural networks. In: Proceedings of the genetic and evolutionary computation conference.

  • Najafabadi MM, Villanustre F, Khoshgoftaar TM, Seliya N, Wald R, Muharemagic E (2015) Deep learning applications and challenges in big data analytics. J Big Data 2:1–21

    Article  Google Scholar 

  • NOAA—National Centers for Environmental Information (2019), viewed 10 January 2021. <www.ncdc.noaa.gov>

  • ONS—Brazilian Independent System Operator (2014) Annual Report of Streamflow Forecasting 2014, viewed 10 January 2021. www.ons.org.br (in portuguese)

  • ONS—Brazilian Independent System Operator (2015) Annual report of streamflow forecasting 2015, viewed 10 January 2021. www.ons.org.br (in portuguese)

  • ONS—Brazilian Independent System Operator (2016) Annual report of streamflow forecasting 2016, viewed 10 January 2021. www.ons.org.br (in portuguese)

  • ONS—Brazilian Independent System Operator (2017) Annual report of streamflow forecasting 2017, viewed 10 January 2021. www.ons.org.br (in portuguese)

  • Polyak BT (1964) Some methods of speeding up the convergence of iteration methods. USSR Comput Math Math Phys 4:1–17

    Article  Google Scholar 

  • Pontes PRM, Fan FM, Fleischmann AS, Rodrigo CDP, Buarque DC, Siqueira VA, Jardim PF, Sorribas MV, Collischonn W (2017) MGB-IPH model for hydrological and hydraulic simulation of large floodplain river systems coupled with open source GIS. Environ Model Softw 94:1–20

    Article  Google Scholar 

  • Quan H, Srinivasan D, Khosravi A (2014) Short-term load and wind power forecasting using neural network-based prediction intervals. IEEE Trans Neural Netw Learn Syst 25:303–315

    Article  Google Scholar 

  • Rasmussen PF, Salas JD, Fagherazzi L, Rassam J, Bobée B (1996) Estimation and validation of contemporaneous PARMA model for streamflow simulation. Water Resour Res 32:3151–3160

    Article  CAS  Google Scholar 

  • REN21 (2018) Renewables 2018 Global Status Report

  • Rozante JR, Moreira DS, Goncalves LGG, Vila DA (2010) Combining TRMM and surface observations of precipitation: technique and validation over South America (2010) Weather and Forecasting 25:885–894

  • Ruder S (2017) An overview of gradient descent optimization algorithms. CoRR. arXiv:1609.04747v2.

  • Russom P (2011) Big data analytics. TDWI best practices report

  • Santos CAG, Silva GBL (2014) Daily streamflow forecasting using a wavelet transform and artificial neural network hybrid models. Hydrol Sci J 59(2):312–324

    Article  CAS  Google Scholar 

  • Schmidhuber J (2015) Deep learning in neural networks: an overview. Neural Netw 61:85–117

    Article  Google Scholar 

  • Sitterson J, Knightes C, Parmar R, Wolfe K, Muche M, Avant B (2017) An overview of rainfall-runoff model types. EPA—United States Environmental Protection Agency

  • Souza FC, Legey LFL (2008) Brazilian electricity market structure and risk management tools. IEEE PES General Meeting.

  • Souza SA, Costa FS, Xavier LNR, Maceira MEP, and Damázio JM (2010) PREVIVAZ—improving weekly streamflow time series forecasts with current hydrologic state of the river basin. International Workshop Advances in Statistical Hydrology

  • Sttari MT, Yurekli K, Pal M (2012) Performance evaluation of artificial neural network approaches in forecasting reservoir inflow. Appl Math Model 36:2649–2657

    Article  Google Scholar 

  • Taormina R, Chau K (2015) Data-driven input variable selection for rainfall-runoff modeling using binary-coded particle swarm optimization and extreme learning. J Hydrol 529:1617–1631

    Article  Google Scholar 

  • TensorFlow (2019) An open-source machine learning framework for everyone. www.tensorflow.org. Accessed 10 Jan 2021 (in portuguese)

  • Tian Y, Pei K, Jama S, and Ray B (2018) DeepTest: automated testing of deep-neural-network-driven autonomous cars. In: ICSE '18 proceedings of the 40th international conference on software engineering, pp 303–314

  • Tongal H, Booij MJ (2018) Simulation and forecasting of streamflows using machine learning models couple with base flow separation. J Hydrol 564:266–282

    Article  Google Scholar 

  • Wan L, Zeiler M, Zhang S, LeCun Y, Fergus R (2013) Regularization of neural networks using drop connect. In: International conference on machine learning (ICML)

  • Witten IH, Frank E, Hall MA, Pal CJ (2017) Data mining practical machine learning tools and techniques, 4th edn. Morgan Kaufmann, Burlington

    Google Scholar 

  • Xu B, Wang N, Chen T, Li M (2015) Empirical evaluation of rectified activations in convolutional network. CoRR, abs/1505.00853

  • Yaseen ZM, El-shafie A, Jaafar O, Afan HA, Sayl KN (2015) Artificial intelligence based model for stream-flow forecasting: 2000–2015. J Hydrol 530:829–844

    Article  Google Scholar 

  • Yaseen ZM, Jaafar O, Deo RC, Kisi O, Adamowski J, Quilty J, El-Shafie A (2016) Stream-flow forecasting using extreme learning machines: a case study in a semi-arid region in Iraq. J Hydrol 542:603–614

    Article  Google Scholar 

  • Zadeh MR, Amin S, Khalili D, Singh VP (2010) Daily outflow prediction by multi layer perceptron with logistic sigmoid and tangent sigmoid activation functions. Water Resour Manag 24:2673–2688

    Article  Google Scholar 

  • Zealand CM, Burn DH, Simonovic SP (1999) Short term streamflow forecasting using artificial neural networks. J Hydrol 214:32–48

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to express their gratitude to AES Tietê for the financial support of the ANEEL Strategic R&D Project 0064-1052/2017. The authors also would like to thank Eduardo H.S. Silva, Lívia M.P. Gazzi, Eliude P. Ferro, and Lanai Torres for technical discussions during the course of this project.

Funding

This research was funded by AES Tietê Brazil under the ANEEL Strategic R&D Project 0064-1052/2017.

Author information

Authors and Affiliations

  1. Operations Research Department, North Carolina State University, Raleigh, NC, USA

    V. A. D. de Faria

  2. Department of Civil, Construction, and Environmental Engineering, North Carolina State University, Raleigh, NC, USA

    A. R. de Queiroz

  3. Duke University’s Nicholas School of the Environment, Duke University, Durham, NC, USA

    L. M. Lima

  4. Department of Electrical Engineering, Federal University of Itajubá, Itajubá, MG, Brazil

    J. W. M. Lima

  5. Institute of Natural Resources, Federal University of Itajubá, Itajubá, MG, Brazil

    B. C. da Silva

Authors
  1. V. A. D. de Faria
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. R. de Queiroz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. L. M. Lima
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. J. W. M. Lima
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. B. C. da Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design. Material preparation, data collection, and analysis were performed by VF and AdQ. The first draft of the manuscript was written by VF, and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to V. A. D. de Faria.

Ethics declarations

Conflict of interest

The authors have no conflicts of interest to declare that are relevant to the content of this article.

Ethics approval

Not applicable.

Consent to participate

Not applicable.

Consent for publication

Not applicable.

Additional information

Editorial responsibility: Maryam Shabani.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (PDF 1265 kb)

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

de Faria, V.A.D., de Queiroz, A.R., Lima, L.M. et al. An assessment of multi-layer perceptron networks for streamflow forecasting in large-scale interconnected hydrosystems. Int. J. Environ. Sci. Technol. 19, 5819–5838 (2022). https://doi.org/10.1007/s13762-021-03565-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13762-021-03565-y

Keywords

search

Navigation