Abstract
The evaluation of precipitation using conventional approaches encounters various challenges stemming from limited financial resources, restricted geographic coverage, and reduced accuracy. In contrast, today urban areas are widely equipped with surveillance cameras, which, combined with advancements in deep learning techniques, has allowed for the creation of a highly effective and precise deep learning model capable of estimating rainfall intensity based on captured videos. In this article, a hybrid regression model based on Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) is developed to estimate rainfall intensity from one-minute videos. Three recurrent units named SimpleRNN, GRU and LSTM have been employed in our hybrid CNN+RNN model, which were tested with Adam, RMSProp and SGD optimizers. In particular, the mean absolute percentage error (MAPE) reports the value between 3.55%-6.95%, which are far better results compared to competitive rainfall estimation models. Also, the proposed model shows a very good performance in terms of complexity by recording the time of 0.47 seconds to calculate 100 frames. This prompt and precise measurement of precipitation is of potential to be applied in effectively managing water resources and mitigating the potential risks of flooding. Our regression model provides rainfall intensity estimates as output in a continuous space, in contrast to some methods, which only categorize the rainfall intensities into some discrete levels. Also, it receives one-minute videos as input, which eliminates the need to interpolate labels to obtain rainfall intensities in multi-millisecond scale as required in image-based algorithms.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
The video dataset used in this work is publicly available at https://figshare.com/articles/dataset/Estimating_rainfall_intensity_with_a_high_spatiotemporal_resolution_using_an_image-based_deep_learning_model/14709423/1.
References
Al-Haija QA, Gharaibeh M, Odeh A (2022) Detection in adverse weather conditions for autonomous vehicles via deep learning. AI 3(2):303–317. https://doi.org/10.3390/ai3020019
Apaydin H, Feizi H, Sattari MT, Colak MS, Shamshirband S, Chau KW (2020) Comparative analysis of recurrent neural network architectures for reservoir inflow forecasting. Water 12(5):1500. https://doi.org/10.3390/w12051500
Bengio Y, Simard P, Frasconi P (1994) Learning long-term dependencies with gradient descent is difficult. IEEE Trans Neural Netw 5(2):157–166
Breinl K, Müller-Thomy H, Blöschl G (2020) Space-time characteristics of areal reduction factors and rainfall processes. J Hydrometeorol 21(4):671–689. https://doi.org/10.1175/JHM-D-19-0228.1
Chen M, Shi W, Xie P, Silva VB, Kousky VE, Wayne Higgins R, Janowiak JE (2008) Assessing objective techniques for gauge-based analyses of global daily precipitation. J Geophys Res Atmos 113(D4). https://doi.org/10.1029/2007JD009132
Chung J, Gulcehre C, Cho K, Bengio Y (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555. https://doi.org/10.48550/arXiv.1412.3555
Dey R, Salem FM (2017) Gate-variants of gated recurrent unit (GRU) neural networks. In 2017 IEEE 60th international Midwest symposium on circuits and systems (MWSCAS), pp 1597–1600. https://doi.org/10.1109/MWSCAS.2017.8053243
Doreswamy, Gad I, Manjunatha BR (2018) Multi-label Classification of Big NCDC Weather Data Using Deep Learning Model. In: 2018 International conference on soft computing systems (ICSCS), communications in computer and information science 837:232–241
Fan B, Xie L, Yang S, Wang L, Soong FK (2016) A deep bidirectional LSTM approach for video-realistic talking head. Multimed Tools Appl 75:5287–5309. https://doi.org/10.1007/s11042-015-2944-3
Gad I, Hosahalli D, Manjunatha BR (2021) A robust deep learning model for missing value imputation in big NCDC dataset. Iran J. Comput. Sci. 4:67–84. https://doi.org/10.1007/s42044-020-00065-z
Germann U, Galli G, Boscacci M, Bolliger M (2006) Radar precipitation measurement in a mountainous region. Q J R Meteorol Soc 132(618):1669–1692. https://doi.org/10.1256/qj.05.190
Gers F (2001) Long short-term memory in recurrent neural networks. Unpublished PhD dissertation, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland
Gers FA, Schmidhuber J, Cummins F (2000) Learning to forget: Continual prediction with LSTM. Neural Comput 12(10):2451–2471
Graves A, Fernández S, Gomez F, Schmidhuber J (2012) Connectionist temporal classification: Supervised sequence labelling with recurrent neural networks. In Proceedings of the 23rd international conference on Machine learning 2006 Jun 25, pp 369–376. https://doi.org/10.1007/978-3-642-24797-2-7
Gulcehre C, Cho K, Pascanu R, Bengio Y (2014) Learned-norm pooling for deep feedforward and recurrent neural networks. In Machine learning and knowledge discovery in databases: European conference, ECML PKDD 2014, Nancy, France, September 15–19, pp 530–546. https://doi.org/10.1007/978-3-662-44848-9-34
Guo Z, Leitao JP, Simões NE, Moosavi V (2021) Data-driven flood emulation: Speeding up urban flood predictions by deep convolutional neural networks. J Flood Risk Manag 14(1):e12684. https://doi.org/10.1111/jfr3.12684
Guo H, Huang H, Sun YE, Zhang Y, Chen S, Huang L (2018) Chaac: Real-time and fine-grained rain detection and measurement using smartphones. IEEE Int Things J 6(1):997–1009. https://doi.org/10.1109/jiot.2018.2866690
Hopfield JJ (1982) Neural networks and physical systems with emergent collective computational abilities. Proc Natl Acad Sci 79(8):2554–2558. https://doi.org/10.1073/pnas.79.8.2554
Jiang S, Babovic V, Zheng Y, Xiong J (2019) Advancing opportunistic sensing in hydrology: A novel approach to measuring rainfall with ordinary surveillance cameras. Water Resour Res 55(4):3004–3027. https://doi.org/10.1029/2018WR024480
Lamichhane N, Sharma S (2017) Development of Flood Warning System and Flood Inundation Mapping Using Field Survey and LiDAR Data for the Grand River near the City of Painesville. Ohio. Hydrology 4(2):24. https://doi.org/10.3390/hydrology4020024
Lin CW, Huang X, Lin M, Hong S (2022) SF-CNN: signal filtering convolutional neural network for precipitation intensity estimation. Sensors 22(02):551. https://doi.org/10.3390/s22020551
Lipton ZC, Berkowitz J, Elkan C (2015) A critical review of recurrent neural networks for sequence learning. arXiv preprint arXiv:1506.00019. https://doi.org/10.48550/arXiv.1506.00019
Madaeni F, Chokmani K, Lhissou R, Homayouni S, Gauthier Y, Tolszczuk-Leclerc S (2022) Convolutional neural network and long short-term memory models for ice-jam predictions. The Cryosphere 16(4):1447–1468. https://doi.org/10.5194/tc-16-1447-2022
Manju D, Seetha M, Sammulal P (2021) Early action prediction using 3DCNN with LSTM and bidirectional LSTM. Turk J Comput Math Educ 12(6):2275–2281
Mu L, Zheng F, Tao R, Zhang Q, Kapelan Z (2020) Hourly and daily urban water demand predictions using a long short-term memory based model. J Water Resour Plan Manag 146(9):05020017. https://doi.org/10.1061/(ASCE)WR.1943-5452.0001276
Orozco CI, Buemi ME, Berlles JJ (2019) CNN-LSTM architecture for action recognition in videos. In I Simposio Argentino de Imagenes y Vision (SAIV 2019)-JAIIO 48 (Salta)
Overeem A, Leijnse H, Uijlenhoet R (2016) Two and a half years of country-wide rainfall maps using radio links from commercial cellular telecommunication networks. Water Resour Res 52(10):8039–8065. https://doi.org/10.1002/2016WR019412
Pyo J, Park LJ, Pachepsky Y, Baek SS, Kim K, Cho KH (2020) Using convolutional neural network for predicting cyanobacteria concentrations in river water. Water Res 186:116349. https://doi.org/10.1016/j.watres.2020.116349
Raissi M, Yazdani A, Karniadakis GE (2020) Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations. Science 367(6481):1026–1030. https://doi.org/10.11262Fscience.aaw4741
Saidi R, Jarray F, Alsuhaibani M (2022) Comparative Analysis of Recurrent Neural Network Architectures for Arabic Word Sense Disambiguation. In Proceedings of the 18th international conference on web information systems and technologies (WEBIST), pp 25–27. https://doi.org/10.5220/0011527600003318
Sangwan N, Bhatnagar V (2021) Video popularity prediction using stacked Bi-LSTM layers. Malays J Comput Sci 34(3):242–254. https://doi.org/10.22452/mjcs.vol34no3.2
Schmidhuber J, Hochreiter S (1997) Long short-term memory. Neural Comput 9(8):1735–1780
Tan M, Le Q (2019) Efficientnet: Rethinking model scaling for convolutional neural networks. In International conference on machine learning, pp 6105–6114
Tian Y, Peters-Lidard CD, Eylander JB, Joyce RJ, Huffman GJ, Adler RF, Zeng J (2009) Component analysis of errors in satellite-based precipitation estimates. J Geophys Res Atmos 114(D24). https://doi.org/10.1029/2009JD011949
TimeDistributed layer (2024) https://keras.io/api/layers/recurrent_layers/time_distributed/
Varga D, Szirányi T (2019) No-reference video quality assessment via pretrained CNN and LSTM networks. SIViP 13:1569–1576. https://doi.org/10.1007/s11760-019-01510-8
Wahl T, Jain S, Bender J, Meyers SD, Luther ME (2015) Increasing risk of compound flooding from storm surge and rainfall for major US cities. Nat Clim Chang 5(12):1093–1097. https://doi.org/10.1038/nclimate2736
Wang C, Hou J, Miller D, Brown I, Jiang Y (2019) Flood risk management in sponge cities: The role of integrated simulation and 3D visualization. Int J Disaster Risk Reduct 39:101139. https://doi.org/10.1016/j.ijdrr.2019.101139
Yan K, Chen H, Hu L, Huang K, Huang Y, Wang Z, Liu B, Wang J, Guo S (2023) A review of video-based rainfall measurement methods. WIREs Water 10(6):e1678. https://doi.org/10.1002/wat2.1678
Yang K, Mall S, Glaser N (2017) Prediction of personality first impressions with deep bimodal LSTM. arXiv preprint arXiv: 1-10
Yin H, Zheng F, Duan HF, Savic D, Kapelan Z (2023) Estimating rainfall intensity using an image-based deep learning model. Engineering 21:162–174. https://doi.org/10.1016/j.eng.2021.11.021
Yuan PH, Yang KF, Tsai WH (2011) Real-time security monitoring around a video surveillance vehicle with a pair of two-camera omni-imaging devices. IEEE Trans Veh Technol 60(8):3603–3614. https://doi.org/10.1109/TVT.2011.2162862
Zaman SM, Hasan MM, Sakline RI, Das D, Alam MA (2021) A comparative analysis of optimizers in recurrent neural networks for text classification. In 2021 IEEE Asia-Pacific conference on computer science and data engineering, pp 1–6. https://doi.org/10.1109/CSDE53843.2021.9718394
Zheng F, Tao R, Maier HR, See L, Savic D, Zhang T, Popescu I (2018) Crowdsourcing methods for data collection in geophysics: State of the art, issues, and future directions. Rev Geophys 56(4):698–740. https://doi.org/10.1029/2018RG000616
Funding
This project received financial support by the Regional Water Company of Qazvin, through grant GZHW2-00001.
Author information
Authors and Affiliations
Contributions
Farshid Rajabi and Neda Faraji wrote and edited the main manuscript, developed the algorithm and implemented it by writing python codes. Masoumeh Hashemi proposed the main idea and the main paper to work on. Also, Neda Faraji revised the manuscript.
Corresponding author
Ethics declarations
Conflicts of interest
The authors declare that they have no conflict of interest.
Additional information
Communicated by: H. Babaie.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Rajabi, F., Faraji, N. & Hashemi, M. An efficient video-based rainfall intensity estimation employing different recurrent neural network models. Earth Sci Inform (2024). https://doi.org/10.1007/s12145-024-01290-x
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s12145-024-01290-x