Skip to main content
SpringerLink
Log in

Time series prediction with hierarchical recurrent model

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

In this paper, we investigate the capability of modeling distant temporal interaction of Long Short-Term Memory (LSTM) and introduce a novel Long Short-Term Memory on time series problems. To increase the capability of modeling distant temporal interactions, we propose a hierarchical architecture (HLSTM) using several LSTM models and a linear layer. This novel framework is then applied to electric power consumption, real-life crime and financial data. We demonstrate in our simulations that this structure significantly improves the modeling of deep temporal connections compared to the classical architecture of LSTM and various studies in the literature. Furthermore, we analyze the sensitivity of the new architecture with respect to the hidden size of LSTM.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Repaka, A.N., Ravikanti, S.D., Franklin, R.G.: Design and implementing heart disease prediction using naives bayesian. In: 2019 3rd International Conference on Trends in Electronics and Informatics (ICOEI), pages 292–297. IEEE, (2019)

  2. Chang, P.C., Fan, C.Y.: A hybrid system integrating a wavelet and tsk fuzzy rules for stock price forecasting. IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.) 38(6), 802–815 (2008)

    Article  Google Scholar 

  3. Anuradhi Malshika Welhenge and Attaphongse Taparugssanagorn: Human activity classification using long short-term memory network. SIViP 13(4), 651–656 (2019)

    Article  Google Scholar 

  4. Koç, E., Türkoğlu, M.: Forecasting of medical equipment demand and outbreak spreading based on deep long short-term memory network: the covid-19 pandemic in turkey. Signal Image Video Process. 16(13), 613–621 (2021)

    Google Scholar 

  5. Hyndman, R.J., Athanasopoulos, G.: Forecasting: principles and practice. OTexts, (2018)

  6. Sharma, Rishi Raj, Kumar, Mohit, Maheshwari, Shishir, Ray, Kamla Prasan: Evdhm-arima-based time series forecasting model and its application for covid-19 cases. IEEE Trans. Instrum. Meas. 70, 1–10 (2020)

    Google Scholar 

  7. Na, Xiaodong, Ren, Weijie, Xinghan, Xu.: Hierarchical delay-memory echo state network: a model designed for multi-step chaotic time series prediction. Eng. Appl. Artif. Intell. 102, 104229 (2021)

    Article  Google Scholar 

  8. Eapen, J., Bein, D., Verma, A.: Novel deep learning model with cnn and bi-directional lstm for improved stock market index prediction. In: 2019 IEEE 9th annual computing and communication workshop and conference (CCWC), pages 0264–0270. IEEE, (2019)

  9. Wu, N., Green, B., Ben, X., O’Banion, S.: Deep transformer models for time series forecasting: The influenza prevalence case. arXiv preprint arXiv:2001.08317, (2020)

  10. Barra, Silvio, Carta, Salvatore Mario, Corriga, Andrea, Podda, Alessandro Sebastian, Recupero, Diego Reforgiato: Deep learning and time series-to-image encoding for financial forecasting. IEEE/CAA J. Autom. Sinica 7(3), 683–692 (2020)

    Article  Google Scholar 

  11. Lindemann, Benjamin, Müller, Timo, Vietz, Hannes, Jazdi, Nasser, Weyrich, Michael: A survey on long short-term memory networks for time series prediction. Procedia CIRP 99, 650–655 (2021)

    Article  Google Scholar 

  12. Huang, J., Zou, X., Fan, Z., Qi, H.: Human pose estimation and lstm-based diver heading prediction for auv navigation guidance. Signal Image Video Process. 16(2), 395–402 (2021)

    Article  Google Scholar 

  13. Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, (2016)http://www.deeplearningbook.org

  14. Aktas, K., Demirel, M., Moor, M., Olesk, J., Ozcinar, C., Anbarjafari, G.: Spatiotemporal based table tennis stroke-type assessment. Signal Image Video Process. 15(7), 1593–1600 (2021)

    Article  Google Scholar 

  15. Ahuja, S., Shelke, N.A., Singh, P.K.: A deep learning framework using cnn and stacked bi-gru for covid-19 predictions in india. Signal Image Video Process. 16(3), 579–586 (2021)

    Article  Google Scholar 

  16. Saroj Kumar Pandey and Rekh Ram Janghel: Automatic arrhythmia recognition from electrocardiogram signals using different feature methods with long short-term memory network model. SIViP 14(6), 1255–1263 (2020)

    Article  Google Scholar 

  17. Effil, N.J., Rajeswari, R.: Wavelet scattering transform and long short-term memory network-based noninvasive blood pressure estimation from photoplethysmograph signals. Signal Image Video Process. 16(1), 1–9 (2021)

    Google Scholar 

  18. Connor, Jerome T., Martin, R Douglas, Atlas, Les E.: Recurrent neural networks and robust time series prediction. IEEE Trans. Neural Netw. 5(2), 240–254 (1994)

    Article  Google Scholar 

  19. Hochreiter, Sepp, Schmidhuber, Jürgen.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  20. Song, Xuanyi, Liu, Yuetian, Xue, Liang, Wang, Jun, Zhang, Jingzhe, Wang, Junqiang, Jiang, Long, Cheng, Ziyan: Time-series well performance prediction based on long short-term memory (lstm) neural network model. J. Petrol. Sci. Eng. 186, 106682 (2020)

    Article  Google Scholar 

  21. Zhao, J., Huang, F., Lv, J., Duan, Y., Qin, Z., Li, G., Tian, G.: Do rnn and lstm have long memory?. In: International Conference on Machine Learning, pages 11365–11375. PMLR, (2020)

  22. Li, Youru, Zhu, Zhenfeng, Kong, Deqiang, Han, Hua, Zhao, Yao: Ea-lstm: evolutionary attention-based lstm for time series prediction. Knowl. Based Syst. 181, 104785 (2019)

    Article  Google Scholar 

  23. Pulver, A., Lyu, S.: Lstm with working memory. In: 2017 International Joint Conference on Neural Networks (IJCNN), pages 845–851. IEEE, (2017)

  24. Pang, Yue, Zhou, Xiangdong, Zhang, Junqi, Sun, Quan, Zheng, Jianbin: Hierarchical electricity time series prediction with cluster analysis and sparse penalty. Pattern Recogn. 126, 108555 (2022)

    Article  Google Scholar 

  25. Xu, M., Han, M., Wang, X.: Hierarchical neural networks for multivariate time series prediction. In: 2016 35th Chinese Control Conference (CCC), pages 6971–6976. IEEE, (2016)

  26. 2012. data retrieved from UCI Machine Learning Repository, https://archive.ics.uci.edu/ml/datasets/individual+household+electric+power+consumption

  27. 2017. data retrieved from Kaggle, https://www.kaggle.com/thebasss/currency-exchange-rates

  28. 2021. data retrieved from Kaggle, https://www.kaggle.com/chicago/chicago-crime

  29. Chujai, P., Kerdprasop, N., Kerdprasop, K.: Time series analysis of household electric consumption with arima and arma models. In: Proceedings of the international multiconference of engineers and computer scientists, volume 1, pages 295–300. IAENG Hong Kong, (2013)

  30. Kim, Tae-Young., Cho, Sung-Bae.: Predicting residential energy consumption using cnn-lstm neural networks. Energy 182, 72–81 (2019)

    Article  Google Scholar 

  31. Le, Tuong, Vo, Minh Thanh, Vo, Bay, Hwang, Eenjun, Rho, Seungmin, Baik, Sung Wook: Improving electric energy consumption prediction using cnn and bi-lstm. Appl. Sci. 9(20), 4237 (2019)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, TOBB University of Economics and Technology, Ankara, 06560, Turkey

    Mustafa Mert Keskin

  2. Department of Computer Engineering, Konya Technical University, Konya, 42250, Turkey

    Fatih Irım

  3. Electrical and Computer Engineering, Carnegie Mellon University, Street, Pittsburgh, Pennsylvania, 15213, USA

    Oğuzhan Karaahmetoğlu

  4. Department of Computer Engineering, Konya Technical University, Konya, 42250, Turkey

    Ersin Kaya

Authors
  1. Mustafa Mert Keskin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fatih Irım
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Oğuzhan Karaahmetoğlu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ersin Kaya
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mustafa Mert Keskin.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Keskin, M.M., Irım, F., Karaahmetoğlu, O. et al. Time series prediction with hierarchical recurrent model. SIViP 17, 2121–2127 (2023). https://doi.org/10.1007/s11760-022-02426-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-022-02426-6

Keywords

Search

Navigation