Abstract
This paper aims to highlight in a relevant way the interest of hybrid models (coupling of ARIMA processes and machine learning models) for economic or financial agents. These models are likely to allow better consideration of certain stylized facts (not necessarily taken into account by often-used models such as ARIMA-GARCH), observed in the analysis of financial time series. Most hybrid models assume that the random perturbation of the chosen ARIMA process follows a normal distribution. Thus, the literature on hybrid models remains on this Gaussian framework and it would be necessary to consider a non-Gaussian and much more realistic framework. We consider hybrid models with the assumption that the random disturbance process follows a Student’s distribution (denoted by ARIMA T), allowing us to take into account the leptokurticity often observed when analyzing financial time series returns. Under certain assumptions, the empirical results show the power of the hybrid models considered.
Similar content being viewed by others
Notes
Loss \(\epsilon \)-insensible function
$$\begin{aligned} L_{\epsilon }(d,y) = \left\{ \begin{array}{ll} |y-f(x)|-\epsilon &{} \text{ si } |y-f(x)| \ge \epsilon \\ 0&{} \text{ sinon }. \end{array} \right. \end{aligned}$$Euclidean norm: ||.||
The most commonly used activation functions are:
-
sigmoid \(\sigma (x) = \frac{1}{1+e^{-x}}\)
-
hyperbolic tangent \(tanh(x) = \frac{e^{x}-e^{-x}}{e^{x}+e^{-x}}\)
-
A stock is a certificate of ownership representing a fractional share in a company’s capital.
Jarque-Bera (JB).
Augmented Dickey-Fuller (ADF).
For more details (see, Kohavi 1995).
References
Agarap, A. F. M. (2018) A neural network architecture combining gated recurrent unit (gru) and support vector machine (svm) for intrusion detection in network traffic data. In Proceedings of the 2018 10th international conference on machine learning and computing (pp. 26–300).
Anastasakis, L., & Mort, N. (2000) Neural network-based prediction of the usd/gbp exchange rate: the utilisation of data compression techniques for input dimension reduction. In Proceedings of the Nostradamus 2000: 3rd conference on prediction, synergetic, behavior of dynamic systems and its application in control theory, physics, chemistry and economy (pp. 3–9).
Andersen, T. G., Bollerslev, T., Diebold, F. X., & Ebens, H. (2001). The distribution of realized stock return volatility. Journal of financial economics, 61(1), 43–76.
Asadi, S., Tavakoli, A., & Hejazi, S. R. (2012). A new hybrid for improvement of auto-regressive integrated moving average models applying particle swarm optimization. Expert Systems with Applications, 39(5), 5332–5337.
Azoff, E. M. (1994) Neural network time series forecasting of financial markets. Wiley.
Bai, S., Kolter, J. Z., & Koltun, V. (2018) An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271
Bernal, A., Fok, S., & Pidaparthi,R. (2012) Financial market time series prediction with recurrent neural networks. State College: Citeseer. [Google Scholar].
Bernardo, J., Berger, J., Dawid, A., & Smith, A., et al. (1998). Regression and classification using gaussian process priors. Bayesian statistics, 6, 475.
Bollerslev, T. (1986). Generalized autoregressive conditional heteroskedasticity. Journal of Econometrics, 31(3), 307–327.
Bollerslev. T. (1987) A conditionally heteroskedastic time series model for speculative prices and rates of return. The Review of Economics and Statistics (pp. 542–547).
Box, G. E., & Jenkins, G. M. (1976) Time series analysis. Forecasting and control. tsaf.
Cao, L., & Tay, F. E. (2001). Financial forecasting using support vector machines. Neural Computing & Applications, 10(2), 184–192.
Cao, Y., Ding, Y., Jia, M., & Tian, R. (2021). A novel temporal convolutional network with residual self-attention mechanism for remaining useful life prediction of rolling bearings. Reliability Engineering & System Safety, 215, 107813.
Cho, K., Van Merriënboer, B., Bahdanau, D., & Bengio, Y. (2014a) On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint arXiv:1409.1259.
Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014b) Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078.
Choi, H. K. (2018) Stock price correlation coefficient prediction with arima-lstm hybrid model. arXiv preprint arXiv:1808.01560.
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555.
Cybenko, G. (1989). Multilayer feedforward networks are universal approximators. Neural Networks, 2(4), 359–366.
Ding, X., Zhang, Y., Liu, T., & Duan, J. (2015) Deep learning for event-driven stock prediction. In Twenty-fourth international joint conference on artificial intelligence.
Engle, R. F. (1982) Autoregressive conditional heteroscedasticity with estimates of the variance of united kingdom inflation. Econometrica: Journal of the Econometric Society, pp. 987–1007.
Fama, E. F. (1995). Random walks in stock market prices. Financial Analysts Journal, 51(1), 75–80.
Fama, E. F., & French, K. R. (1992). The cross-section of expected stock returns. The Journal of Finance, 47(2), 427–465.
Fathi, O. (2019) Time series forecasting using a hybrid arima and lstm model. In Velvet consulting.
Fischer, T., & Krauss, C. (2018). Deep learning with long short-term memory networks for financial market predictions. European Journal of Operational Research, 270(2), 654–669.
Glosten, L. R., Jagannathan, R., & Runkle, D. E. (1993). On the relation between the expected value and the volatility of the nominal excess return on stocks. The Journal of Finance, 48(5), 1779–1801.
Gourinchas, P.-O., & Tornell, A. (2004). Exchange rate puzzles and distorted beliefs. Journal of International Economics, 64(2), 303–333.
Granger, C. (1989). Combining forecasts-twenty years later. Journal of Forecasting, 8, 167–173.
Granger, C. W. (1969) Investigating causal relations by econometric models and cross-spectral methods. Econometrica: Journal of the Econometric Society pp. 424–438.
Granger, C. W. (1992). Forecasting stock market prices: Lessons for forecasters. International Journal of Forecasting, 8(1), 3–13.
Granger, C. W. J. (1991) Modelling economic series: readings in econometric methodology. Oxford: Oxford University Press.
Greff, K., Srivastava, R. K., Koutník, J., Steunebrink, B. R., & Schmidhuber, J. (2016). Lstm: A search space odyssey. IEEE Transactions on Neural Networks and Learning Systems, 28(10), 2222–2232.
Guo, H., Liu, X., & Sun, Z. (2016). Multivariate time series prediction using a hybridization of varma models and Bayesian networks. Journal of Applied Statistics, 43(16), 2897–2909.
He, K., Zhang, X., Ren, S., & Sun, J. (2016) Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770–778).
Hernández-Lobato, J. M., & Adams, R. (2015) Probabilistic backpropagation for scalable learning of Bayesian neural networks. In International conference on machine learning (pp. 1861–1869). PMLR.
Hewage, P., Behera, A., Trovati, M., Pereira, E., Ghahremani, M., Palmieri, F., & Liu, Y. (2020). Temporal convolutional neural (tcn) network for an effective weather forecasting using time-series data from the local weather station. Soft Computing, 24, 16453–16482.
Hipel, K. W., & McLeod, A. I. (1994) Time series modelling of water resources and environmental systems. Elsevier
Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780.
Huang, W., Lai, K. K., Nakamori, Y., & Wang, S. (2004). Forecasting foreign exchange rates with artificial neural networks: A review. International Journal of Information Technology & Decision Making, 3(01), 145–165.
Huang, W., Nakamori, Y., & Wang, S.-Y. (2005). Forecasting stock market movement direction with support vector machine. Computers & Operations Research, 32(10), 2513–2522.
Ji, W., & Chee, K. C. (2011). Prediction of hourly solar radiation using a novel hybrid model of arma and tdnn. Solar Energy, 85(5), 808–817.
Kamdem, J. S., Essomba, R. B., & Berinyuy, J. N. (2020). Deep learning models for forecasting and analyzing the implications of covid-19 spread on some commodities markets volatilities. Chaos, Solitons & Fractals, 140, 110215.
Kia, A. N., Fathian, M., & Gholamian, M. (2012). Using mlp and rbf neural networks to improve the prediction of exchange rate time series with arima. International Journal of Information and Electronics Engineering, 2(4), 543–546.
Kim, S.-J., & Sheen, J. (2006). Interventions in the yen-dollar spot market: A story of price, volatility and volume. Journal of Banking & Finance, 30(11), 3191–3214.
Kohavi, R., et al. (1995). A study of cross-validation and bootstrap for accuracy estimation and model selection. Ijcai, 14, 1137–1145.
Krogh,A., & Vedelsby, J. (1994) Neural network ensembles, cross validation, and active learning. Advances in Neural Information Processing Systems, 7.
Kumar, M., & Thenmozhi, M. (2014). Forecasting stock index returns using arima-svm, arima-ann, and arima-random forest hybrid models. International Journal of Banking, Accounting and Finance, 5(3), 284–308.
Lawrence, N., & Hyvärinen, A. (2005) Probabilistic non-linear principal component analysis with gaussian process latent variable models. Journal of Machine Learning Research, 6(11).
LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278–2324.
Li, T., Hua, M., & Wu, X. (2020). A hybrid cnn-lstm model for forecasting particulate matter (pm2. 5). Ieee Access, 8, 26933–26940.
Livieris, I. E., Pintelas, E., & Pintelas, P. (2020). A cnn-lstm model for gold price time-series forecasting. Neural Computing and Applications, 32(23), 17351–17360.
Lo, A. W., & MacKinlay, A. C. (1988). Stock market prices do not follow random walks: Evidence from a simple specification test. The Review of Financial Studies, 1(1), 41–66.
Lo, A. W., Mamaysky, H., & Wang, J. (2000). Foundations of technical analysis: Computational algorithms, statistical inference, and empirical implementation. The Journal of Finance, 55(4), 1705–1765.
Lu, W., Li, J., Li, Y., Sun, A., & Wang, J. (2020). A cnn-lstm-based model to forecast stock prices. Complexity, 2020.
MacKay, D. J., et al. (1998). Introduction to gaussian processes. NATO ASI Series F Computer and Systems Sciences, 168, 133–166.
Malkiel, B. G. (1999) A random walk down Wall Street: including a life-cycle guide to personal investing. WW Norton & Company.
McKenzie, C., & Hipel, K. (1994). The gamma arima (garima) model for irregularly observed time series. Journal of Time Series Analysis, 15(5), 541–553.
Mikosch, T., & Stărică, C. (2004). Nonstationarities in financial time series, the long-range dependence, and the igarch effects. Review of Economics and Statistics, 86(1), 378–390.
Moghadam,R. A., & Keshmirpour, M. (2011) Hybrid arima and neural network model for measurement estimation in energy-efficient wireless sensor networks. In International conference on informatics engineering and information science (pp. 35–48) Springer.
Mukherjee, S., Osuna, E., & Girosi, F. (1997) Nonlinear prediction of chaotic time series using support vector machines. In Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Signal Processing Society Workshop, pages 511–520. IEEE
Müller, K.-R., Smola, A. J., Rätsch, G., Schölkopf, B., Kohlmorgen, J., & Vapnik, V. (1997) Predicting time series with support vector machines. In International conference on artificial neural networks (pp. 999–1004). Springer.
Nash, J. E., & Sutcliffe, J. V. (1970). River flow forecasting through conceptual models part i-a discussion of principles. Journal of Hydrology, 10(3), 282–290.
Navon, A., & Keller, Y. (2017) Financial time series prediction using deep learning. arXiv preprint arXiv:1711.04174
Nawi, W. I. A. W. M., Lola, M. S., Zakariya, R., Zainuddin, N. H., Abd Hamid, A. A. K., Aruchunan, E., & Nazzrol, N. S. A. (2021). Improved of forecasting sea surface temperature based on hybrid arima and support vector machines models. Malaysian Journal of Fundamental and Applied Sciences, 17(5), 609–620.
Nelson, D. B. (1991) Conditional heteroskedasticity in asset returns: A new approach. Econometrica: Journal of the Econometric Society 347–370.
Nelson, D. M., Pereira, A. C., & de Oliveira, R. A. (2017) Stock market’s price movement prediction with lstm neural networks. In 2017 International joint conference on neural networks (IJCNN) (pp.1419–1426). IEEE.
Ordóñez, C., Lasheras, F. S., Roca-Pardiñas, J., & de Cos Juez, F. J. (2019). A hybrid arima-svm model for the study of the remaining useful life of aircraft engines. Journal of Computational and Applied Mathematics, 346, 184–191.
Oreshkin, B. N., Carpov, D., Chapados, N., & Bengio, Y. (2019) N-beats: Neural basis expansion analysis for interpretable time series forecasting. arXiv preprint arXiv:1905.10437
Pai, P.-F., & Lin, C.-S. (2005). A hybrid arima and support vector machines model in stock price forecasting. Omega, 33(6), 497–505.
Pan, F., Zhang, H., & Xia, M. (2009) A hybrid time-series forecasting model using extreme learning machines. In 2009 second international conference on intelligent computation technology and automation (vol. 1, pp. 933–936). IEEE.
Pesaran, M. H. (2015). Time series and panel data econometrics. Oxford: Oxford University Press.
Rajalakshmi, V., et al. (2022). Hybrid time-series forecasting models for traffic flow prediction. Promet-Traffic & Transportation, 34(4), 537–549.
Rasmussen, C. E., & Williams, C. K. (2006). et al. Gaussian processes for machine learning (vol. 1. Springer, 2006).
Rocha, A. V., & Cribari-Neto, F. (2009). Beta autoregressive moving average models. Test, 18(3), 529.
Roondiwala, M., Patel, H., & Varma, S. (2017). Predicting stock prices using lstm. International Journal of Science and Research (IJSR), 6(4), 1754–1756.
Rosenblatt, F. (1957). The perceptron, a perceiving and recognizing automaton Project Para. Cornell Aeronautical Laboratory.
Rosenblatt, F. (1961). Principles of neurodynamics. Perceptrons and the theory of brain mechanisms. Technical report, Cornell Aeronautical Lab Inc Buffalo NY.
Sen, R., Yu, H.-F., Dhillon, I. S. (2019) Think globally, act locally: A deep neural network approach to high-dimensional time series forecasting. Advances in Neural Information Processing Systems, 32.
Sharpe, W. F. (1964). Capital asset prices: A theory of market equilibrium under conditions of risk. The Journal of Finance, 19(3), 425–442.
Shepard, N. (1996). Statistical aspects of arch and stochastic volatility. Finance and other Fields: Time Series Models in Econometrics.
Shiller, R. J. (1980) Do stock prices move too much to be justified by subsequent changes in dividends?, 1980.
Simon, P. (2013) Too big to ignore: the business case for big data (vol. 72.) John Wiley & Sons.
Smola, A. J., & Schölkopf, B. (2004). A tutorial on support vector regression. Statistics and Computing, 14(3), 199–222.
Srivastava, R. K., Greff, K., & Schmidhuber, J. (2015) Training very deep networks. Advances in Neural Information Processing Systems, 28.
Sutskever, I., Vinyals, O., & Le,Q. V. (2014) Sequence to sequence learning with neural networks. Advances in Neural Information Processing Systems, 27.
Tay, F. E., & Cao, L. (2001) Application of support vector machines in financial time series forecasting. omega, 29(4), 309–317.
Temur, A. S., Akgün, M., & Temur,G. (2019) Predicting housing sales in turkey using arima, lstm and hybrid models.
Teräsvirta, T., Lin, C.-F., & Granger, C. W. (1993). Power of the neural network linearity test. Journal of Time Series Analysis, 14(2), 209–220.
Tiku, M. L., Wong, W.-K., Vaughan, D. C., & Bian, G. (2000). Time series models in non-normal situations: Symmetric innovations. Journal of Time Series Analysis, 21(5), 571–596.
Trippi, R. R., & DeSieno, D. (1992). Trading equity index futures with a neural network. Journal of Portfolio Management, 19, 27–27.
Tsay, R. S. (2005) Analysis of financial time series. John wiley & sons.
Vapnik, V. N. (1995) The nature of statistical learning. Theory.
Vidal, A., & Kristjanpoller, W. (2020). Gold volatility prediction using a cnn-lstm approach. Expert Systems with Applications, 157, 113481.
Waibel, A., Hanazawa, T., Hinton, G., Shikano, K., & Lang, K. J. (1989). Phoneme recognition using time-delay neural networks. IEEE Transactions on Acoustics, Speech, and Signal Processing, 37(3), 328–339.
Walker, G. T. (1931). On periodicity in series of related terms. Proceedings of the Royal Society of London Series A Containing Papers of a Mathematical and Physical Character, 131(818), 518–532.
Wan, R., Mei, S., Wang, J., Liu, M., & Yang, F. (2019). Multivariate temporal convolutional network: A deep neural networks approach for multivariate time series forecasting. Electronics, 8(8), 876.
Welling, M. (2004). Support vector regression. Department of Computer Science: University of Toronto, Toronto (Kanada).
Williams, C. K. (1998) Prediction with gaussian processes: From linear regression to linear prediction and beyond. Learning in Graphical Models (pp. 599–621).
Williams, C. K., & Rasmussen, C. E. (2006). Gaussian processes for machine learning (Vol. 2). Cambridge, MA: MIT Press.
Wilson, A., & Adams, R. (2013) Gaussian process kernels for pattern discovery and extrapolation. In International conference on machine learning (pp. 1067–1075). PMLR.
Xiong, R., Nichols, E. P., & Shen, Y. (2015) Deep learning stock volatility with google domestic trends. arXiv preprint arXiv:1512.04916
Xu, D., Zhang, Q., Ding, Y., & Huang, H. (2020). Application of a hybrid arima-svr model based on the spi for the forecast of drought-a case study in henan province, china. Journal of Applied Meteorology and Climatology, 59(7), 1239–1259.
Xu, D., Zhang, Q., Ding, Y., & Zhang, D. (2022). Application of a hybrid arima-lstm model based on the spei for drought forecasting. Environmental Science and Pollution Research, 29(3), 4128–4144.
Yosinski, J., Clune, J., Nguyen, A., Fuchs, T., & Lipson, H. (2015) Understanding neural networks through deep visualization. arXiv preprint arXiv:1506.06579
Yu, B., Yin, H., & Zhu,Z. (2017a) Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting. arXiv preprint arXiv:1709.04875.
Yu, F., & Koltun, V. (2015) Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122
Yu, F., Koltun, V., & Funkhouser,T. (2017b) Dilated residual networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 472–480).
Yule, G. U. (1927). Vii on a method of investigating periodicities disturbed series, with special reference to wolfer’s sunspot numbers. Philosophical Transactions of the Royal Society of London Series A, Containing Papers of a Mathematical or Physical Character, 226(636–646), 267–298.
Zhang, C., Wei, H., Zhao, X., Liu, T., & Zhang, K. (2016). A gaussian process regression based hybrid approach for short-term wind speed prediction. Energy Conversion and Management, 126, 1084–1092.
Zhang, G. P. (2003). Time series forecasting using a hybrid arima and neural network model. Neurocomputing, 50, 159–175.
Zhang, T., & Benveniste, A. (1992). On the convergence properties of the gradient method for training multilayer feedforward neural networks. Neural Networks, 5(2), 159–175.
Zhu, B., Chevallier, J., Zhu, B., & Chevallier, J. (2017) Carbon price forecasting with a hybrid arima and least squares support vector machines methodology. Pricing and Forecasting Carbon Markets: Models and Empirical Analyses 87–107.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing interest
All authors declare that they have no conflicts of interest.
Ethical approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Pokou, F., Sadefo Kamdem, J. & Benhmad, F. Hybridization of ARIMA with Learning Models for Forecasting of Stock Market Time Series. Comput Econ 63, 1349–1399 (2024). https://doi.org/10.1007/s10614-023-10499-9
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10614-023-10499-9