Skip to main content
SpringerLink
Log in

Lagging problem in financial time series forecasting

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Accurate financial time series forecasting is important in financial markets. However, for financial time series with low fluctuation, there is an unusual forecasting phenomenon in the popular recurrent network model forecasting, with the predictive value lagging the truth value. We call this phenomenon the lagging problem. This study proposes new evaluation measures for assessing the lagging problem, including lagging relative error, lagging value error, and lagging trend error. Moreover, the state analysis method and linear fitting model are developed to explain the causes of the lagging problem. Experimental results show that all popular recurrent network models adopted suffer from the lagging problem. This problem is caused by the failure of the nonlinear function in the prediction model and the linear degeneration of the prediction model thereafter, resulting in the suppression of the nonlinear fitting ability.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Availability of data and materials

The datasets supporting the conclusions of this article are included within the article and its additional files.

References

  1. Yolcu U, Egrioglu E, Aladag CH (2013) A new linear & nonlinear artificial neural network model for time series forecasting. Decis Support Syst 54(3):1340–1347. https://doi.org/10.1016/j.dss.2012.12.006

    Article  Google Scholar 

  2. Parida A, Bisoi R, Dash P (2016) Chebyshev polynomial functions based locally recurrent neuro-fuzzy information system for prediction of financial and energy market data. J Financ Data Sci 2(3):202–223. https://doi.org/10.1016/j.jfds.2016.10.001

    Article  Google Scholar 

  3. Cheng C-H, Yang J-H (2018) Fuzzy time-series model based on rough set rule induction for forecasting stock price. Neurocomputing 302:33–45. https://doi.org/10.1016/j.neucom.2018.04.014

    Article  Google Scholar 

  4. Shen Z, Wang W, Shen Q et al (2020) A novel learning method for multi-intersections aware traffic flow forecasting. Neurocomputing 398:477–484. https://doi.org/10.1016/j.neucom.2019.04.094

    Article  Google Scholar 

  5. Contreras-Reyes JE, Idrovo-Aguirre BJ (2020) Backcasting and forecasting time series using detrended cross-correlation analysis. Phys A Stat Mech Appl 560:125109. https://doi.org/10.1016/j.physa.2020.125109

    Article  MathSciNet  MATH  Google Scholar 

  6. LeCun Y, Bengio Y, Hinton G (2015) Deep learning nature 521(7553):436–444. https://doi.org/10.1038/nature14539

    Article  Google Scholar 

  7. Sezer OB, Gudelek MU, Ozbayoglu AM (2020) Financial time series forecasting with deep learning: a systematic literature review. Appl Soft Comput 90:106181. https://doi.org/10.1016/j.asoc.2020.106181

    Article  Google Scholar 

  8. Chong E, Han C, Park FC (2017) Deep learning networks for stock market analysis and prediction: methodology, data representations, and case studies. Expert Syst Appl 83:187–205. https://doi.org/10.1016/j.eswa.2017.04.030

    Article  Google Scholar 

  9. Hiransha M, Gopalakrishnan EA, Menon VK et al (2018) NSE stock market prediction using deep-learning models. Procedia Comput Sci 132:1351–1362. https://doi.org/10.1016/j.procs.2018.05.050

    Article  Google Scholar 

  10. Jeong G, Kim HY (2019) Improving financial trading decisions using deep Q-learning: predicting the number of shares, action strategies, and transfer learning. Expert Syst Appl 117:125–138. https://doi.org/10.1016/j.eswa.2018.09.036

    Article  Google Scholar 

  11. Chen W, Yeo CK, Lau CT et al (2018) Leveraging social media news to predict stock index movement using RNN-boost. Data Knowl Eng 118:14–24. https://doi.org/10.1016/j.datak.2018.08.003

    Article  Google Scholar 

  12. Deng Y, Bao F, Kong Y et al (2016) Deep direct reinforcement learning for financial signal representation and trading. IEEE Trans Neural Netw Learn Syst 28(3):653–664. https://doi.org/10.1109/TNNLS.2016.2522401

    Article  Google Scholar 

  13. Wang B, Wang J (2019) Energy futures prices forecasting by novel DPFWR neural network and DS-CID evaluation. Neurocomputing 338:1–15. https://doi.org/10.1016/j.neucom.2019.01.092

    Article  MathSciNet  Google Scholar 

  14. Baffour AA, Feng J, Taylor EK (2019) A hybrid artificial neural network-GJR modeling approach to forecasting currency exchange rate volatility. Neurocomputing 365:285–301. https://doi.org/10.1016/j.neucom.2019.07.088

    Article  Google Scholar 

  15. Sherstinsky A (2020) Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network. Phys D: Nonlinear Phenom 404:132306. https://doi.org/10.1016/j.physd.2019.132306

    Article  MathSciNet  MATH  Google Scholar 

  16. Shen G, Tan Q, Zhang H et al (2018) Deep learning with gated recurrent unit networks for financial sequence predictions. Procedia Comput Sci 131:895–903. https://doi.org/10.1016/j.procs.2018.04.298

    Article  Google Scholar 

  17. Bengio Y, Simard P, Frasconi P (1994) Learning long-term dependencies with gradient descent is difficult. IEEE Trans Neural Netw 5(2):157–166. https://doi.org/10.1109/72.279181

    Article  Google Scholar 

  18. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735

    Article  Google Scholar 

  19. Graves A (2013) Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850. doi:https://doi.org/10.48550/arXiv.1308.0850

  20. Cho K, Van Merriënboer B, Bahdanau D et al (2014) On the properties of neural machine translation: encoder–decoder approaches. arXiv preprint arXiv:1409.1259. doi:https://doi.org/10.48550/arXiv.1409.1259

  21. Chung J, Gulcehre C, Cho K et al (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555. doi:https://doi.org/10.48550/arXiv.1412.3555

  22. Berradi Z, Lazaar M (2018) Integration of principal component analysis and recurrent neural network to forecast the stock price of casablanca stock exchange. In: 2nd International Conference on Intelligent Computing in Data Sciences (ICDS), pp 55–61. doi:https://doi.org/10.1016/j.procs.2019.01.008

  23. Lu ZC, Long W, Guo Y (2018) Extreme market prediction for trading signal with deep recurrent neural network. In: 18th International Conference on Computational Science (ICCS), pp 410–418. doi:https://doi.org/10.1007/978-3-319-93701-4_31

  24. Dixon M (2018) Sequence classification of the limit order book using recurrent neural networks. J Comput Sci 24:277–286. https://doi.org/10.1016/j.jocs.2017.08.018

    Article  MathSciNet  Google Scholar 

  25. Yeung J, Wei ZK, Chan KY et al (2020) Jump detection in financial time series usingmachine learning algorithms. Soft Comput 24(3):1789–1801. https://doi.org/10.1007/s00500-019-04006-2

    Article  Google Scholar 

  26. Co NT, Son HH, Hoang NT et al (2020) Comparison between ARIMA and LSTM-RNN for VN-index prediction. In: 3rd International Conference on Intelligent Human Systems Integration (IHSI) - Integrating People and Intelligent Systems, pp 1107–1112. doi:https://doi.org/10.1007/978-3-030-39512-4_168

  27. Sutradhar K, Sutradhar S, Jhimel IA et al (2021) Stock market prediction using recurrent neural network’s LSTM architecture. In: 12th IEEE Annual Ubiquitous Computing, Electronics and Mobile Communication Conference (UEMCON), pp 541–547. doi:https://doi.org/10.1109/uemcon53757.2021.9666562

  28. Assaf O, Di Fatta G, Nicosia G (2021) Multivariate LSTM for stock market volatility prediction. In: 7th International Conference on Machine Learning, Optimization, and Data Science (LOD) / 1st Symposium on Artificial Intelligence and Neuroscience (ACAIN), pp 531–544. doi:https://doi.org/10.1007/978-3-030-95470-3_40

  29. Kumar R, Kumar P, Kumar Y et al (2021) Analysis of financial time series forecasting using deep learning model. In: 11th International Conference on Cloud Computing, Data Science and Engineering (Confluence), pp 877–881. doi:https://doi.org/10.1109/Confluence51648.2021.9377158

  30. Li JM, Wang J (2020) Forcasting of energy futures market and synchronization based on stochastic gated recurrent unit model. Energy. https://doi.org/10.1016/j.energy.2020.118787

    Article  Google Scholar 

  31. Ungureanu S, Topa V, Cziker AC (2021) Analysis for non-residential short-term load forecasting using machine learning and statistical methods with financial impact on the power market. Energies 14(21):6966. https://doi.org/10.3390/en14216966

    Article  Google Scholar 

  32. Touzani Y, Douzi K (2021) An LSTM and GRU based trading strategy adapted to the Moroccan market. J Big Data. https://doi.org/10.1186/s40537-021-00512-z

    Article  Google Scholar 

  33. Alqahtani AS, Kshirsagar PR, Manoharan H et al (2022) Prophetic energy assessment with smart implements in hydroelectricity entities using artificial intelligence algorithm. Int Trans Electr Energy Syst. https://doi.org/10.1155/2022/2376353

    Article  Google Scholar 

  34. Yan X, Wang WH, Chang M (2021) Research on financial assets transaction prediction model based on LSTM neural network. Neural Comput Appl 33(1):257–270. https://doi.org/10.1007/s00521-020-04992-7

    Article  Google Scholar 

  35. Banerjee T, Sinha S, Choudhury P (2022) Long term and short term forecasting of horticultural produce based on the LSTM network model. Appl Intell 52(8):9117–9147. https://doi.org/10.1007/s10489-021-02845-x

    Article  Google Scholar 

  36. Yang C, Guo SH (2021) Inflation prediction method based on deep learning. Comput Intell Neurosci. https://doi.org/10.1155/2021/1071145

    Article  Google Scholar 

  37. Gupta U, Bhattacharjee V, Bishnu PS (2022) StockNet-GRU based stock index prediction. Expert Syst Appl. https://doi.org/10.1016/j.eswa.2022.117986

    Article  Google Scholar 

  38. Nelson CR, Plosser CR (1982) Trends and random walks in macroeconmic time series: some evidence and implications. J Monet Econ 10(2):139–162. https://doi.org/10.1016/0304-3932(82)90012-5

    Article  Google Scholar 

  39. Sagheer A, Kotb M (2019) Time series forecasting of petroleum production using deep LSTM recurrent networks. Neurocomputing 323:203–213. https://doi.org/10.1016/j.neucom.2018.09.082

    Article  Google Scholar 

  40. Lin YF, Huang TM, Chung WH et al (2021) Forecasting fluctuations in the financial index using a recurrent neural network based on price features. IEEE Trans Emerg Top Comput Intell 5(5):780–791. https://doi.org/10.1109/tetci.2020.2971218

    Article  Google Scholar 

  41. Wang L, Gupta S (2013) Neural networks and wavelet de-noising for stock trading and prediction. In: Pedrycz W, Chen S-M (eds) Time series analysis, modeling and applications: a computational intelligence perspective. Springer Berlin Heidelberg, Berlin, Heidelberg, pp 229–247. https://doi.org/10.1007/978-3-642-33439-9_11

    Chapter  Google Scholar 

  42. Hajiabotorabi Z, Kazemi A, Samavati FF et al (2019) Improving DWT-RNN model via B-spline wavelet multiresolution to forecast a high-frequency time series. Expert Syst Appl. https://doi.org/10.1016/j.eswa.2019.112842

    Article  Google Scholar 

  43. Yan H, Ouyang H (2018) Financial time series prediction based on deep learning. Wirel Pers Commun 102(2):683–700. https://doi.org/10.1007/s11277-017-5086-2

    Article  Google Scholar 

  44. Huang NE, Shen Z, Long SR et al (1998) The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis. Proc Royal Soc a-Math Phys Eng Sci 454(1971):903–995. https://doi.org/10.1098/rspa.1998.0193

    Article  MathSciNet  MATH  Google Scholar 

  45. Wu Z, Huang NE (2009) Ensemble empirical mode decomposition: a noise-assisted data analysis method. Adv Adapt Data Anal 1(01):1–41. https://doi.org/10.1142/S1793536909000047

    Article  Google Scholar 

  46. Torres ME, Colominas MA, Schlotthauer G et al (2011) A complete ensemble empirical mode decomposition with adaptive noise. In: 2011 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 4144–4147. doi:https://doi.org/10.1109/ICASSP.2011.5947265

  47. He J (2021) Application of deep learning model under improved emd in railway transportation investment benefits and national economic attribute analysis. J Supercomput 77(8):8194–8208. https://doi.org/10.1007/s11227-020-03609-z

    Article  Google Scholar 

  48. Lin HL, Sun QB (2020) Crude oil prices forecasting: an approach of using CEEMDAN-based multi-layer gated recurrent unit networks. Energies. https://doi.org/10.3390/en13071543

    Article  Google Scholar 

  49. Yang M, Wang J (2022) Adaptability of financial time series prediction based on BiLSTM. Procedia Comput Sci 199:18–25. https://doi.org/10.1016/j.procs.2022.01.003

    Article  Google Scholar 

  50. Akbar SB, Thanupillai K, Govindarajan V (2022) Forecasting Bitcoin price using time opinion mining and bi-directional GRU. J Intell Fuzzy Syst 42(3):1825–1833. https://doi.org/10.3233/jifs-211217

    Article  Google Scholar 

  51. Hyndman RJ, Koehler AB (2006) Another look at measures of forecast accuracy. Int J Forecast 22(4):679–688. https://doi.org/10.1016/j.ijforecast.2006.03.001

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Intelligent Machines, Hefei Institutes of Physical Science, Chinese Academy of Sciences, Hefei, 230031, China

    Jincheng Li, Liangtu Song, Di Wu, Jiahao Shui & Tao Wang

  2. University of Science and Technology of China, Hefei, 230026, China

    Jincheng Li, Di Wu & Jiahao Shui

  3. Institute of Physical Science and Information Technology, Anhui University, Hefei, 230039, China

    Tao Wang

Authors
  1. Jincheng Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Liangtu Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Di Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jiahao Shui
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jincheng Li.

Ethics declarations

Conflict of interest

The authors have no competing interests to declare that are relevant to the content of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, J., Song, L., Wu, D. et al. Lagging problem in financial time series forecasting. Neural Comput & Applic 35, 20819–20839 (2023). https://doi.org/10.1007/s00521-023-08879-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-08879-1

Keywords

Search

Navigation