Advertisement

International Journal of Fuzzy Systems

, Volume 20, Issue 3, pp 701–728 | Cite as

A New Approach for Time Series Prediction Using Ensembles of IT2FNN Models with Optimization of Fuzzy Integrators

  • Jesus Soto
  • Patricia Melin
  • Oscar CastilloEmail author
Article

Abstract

This paper describes a new approach for time series prediction based on using different soft computing techniques, such as neural networks (NNs), type-1 and type-2 fuzzy logic systems and bio-inspired algorithms, where each of these intelligent techniques can provide a variety of features for solving real and complex problems. Therefore, this paper describes the application of ensembles of interval type-2 fuzzy neural network (IT2FNN) models. The IT2FNN uses hybrid learning algorithm techniques from NNs models and fuzzy logic systems. The output of the Ensemble of IT2FNN models needs the integration process to forecast the time series, and we are required to design the fuzzy integrator (FI) to solve this real problem. Genetic algorithms and particle swarm optimization are used for the optimization of the parameter values in the membership functions of the FI. We consider different time series to measure the performance of the proposed model, and these time series are: Mackey–Glass, Mexican Stock Exchange (MSE or BMV), Dow Jones and NASDAQ. The forecasting errors are calculated as follows: mean absolute error, mean square error (MSE), root-mean-square error, mean percentage error and mean absolute percentage error. The best prediction errors are illustrated as follows: 0.00025 for the Mackey–Glass, 0.01012 for the MSE, 0.01307 for the Dow Jones and 0.01171 for the NASDAQ time series. Simulation results are compared using a statistical test and provide evidence of the potential advantages of the proposed approach.

Keywords

Time series Ensembles IT2FNN Fuzzy integrators Genetic algorithm Particle swarm optimization Statistical test 

References

  1. 1.
    Rather, A.M., Agarwal, A., Sastry, V.N.: Recurrent neural network and a hybrid model for prediction of stock returns. Exp. Syst. Appl. 42(5), 3234–3241 (2015)CrossRefGoogle Scholar
  2. 2.
    Ascia, G., Catania, V., Panno, D.: An integrated fuzzy-GA approach for buffer management. IEEE Trans. Fuzzy Syst. 14(4), 528–541 (2006)CrossRefGoogle Scholar
  3. 3.
    Blau, B.M., Van-Ness, B.F., Van-Ness, R.A.: Information in short selling: comparing NASDAQ and the NYSE. Rev. Financ. Econ. 20(1), 1–10 (2011)CrossRefGoogle Scholar
  4. 4.
    Bonissone, P.P., Subbu, R., Eklund, N., Kiehl, T.R.: Evolutionary algorithms + domain knowledge = real-world evolutionary computation. IEEE Trans. Evol. Comput. 10(3), 256–280 (2006)CrossRefGoogle Scholar
  5. 5.
    Box, G.E.P., Jenkins, G.M., Reinsel, G.C.: Time Series Analysis: Forecasting and Control, 3rd edn. Prentice Hall, Englewood Cliffs (1994)zbMATHGoogle Scholar
  6. 6.
    Buckles, B.P., Petry, F.E.: Genetic Algorithms. IEEE Computer Society Press, Washington (1992)zbMATHGoogle Scholar
  7. 7.
    Castellanos, S.G., Martínez, L.: Development of the Mexican bond market. In: Borensztein, E., Cowan, K., Eichengreen, B., Panizza, U. (eds.) Bond Markets in Latin America: On the Verge of a Big Bang?, pp. 51–58. MIT Press, Cambridge (2008)Google Scholar
  8. 8.
    Castillo, O., Melin, P.: Comparison of Hybrid Intelligent Systems, Neural Networks and Interval Type-2 Fuzzy Logic for Time Series Prediction, pp. 3086–3091. IJCNN, Orlando (2007)Google Scholar
  9. 9.
    Castillo, O., Melin, P.: Optimization of type-2 fuzzy systems based on bio-inspired methods: a concise review. Inf. Sci. 205, 1–19 (2012)CrossRefGoogle Scholar
  10. 10.
    Castro, J.R., Castillo, O., Martínez, L.G.: Interval type-2 fuzzy logic toolbox. Eng. Lett. 15, 89–98 (2007)Google Scholar
  11. 11.
    Castro, J.R., Castillo, O., Melin, P., Rodriguez, A.: A Hybrid Learning Algorithm for Interval Type-2 Fuzzy Neural Networks: The Case of Time Series Prediction, vol. 15a, pp. 363–386. Springer, Berlin (2008)Google Scholar
  12. 12.
    Chiou, Y.C., Lan, L.W.: Genetic fuzzy logic controller: an iterative evolution algorithm with new encoding method. Fuzzy Sets Syst. 152, 617–635 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Clerc, M., Kennedy, J.: The particle swarm-explosion, stability, and convergence in a multimodal complex space. IEEE Trans. Evol. Comput. 6, 58–73 (2002)CrossRefGoogle Scholar
  14. 14.
    Cowpertwait, P.S.P., Metcalfe, A.V.: Introductory Time Series with R, pp. 2–5. Springer, Dordrecht (2009)zbMATHGoogle Scholar
  15. 15.
    Deb, K.: A population-based algorithm-generator for real-parameter optimization. Soft Comput. 9(4), 236–253 (2005)CrossRefzbMATHGoogle Scholar
  16. 16.
    Dow Jones Company. http://www.dowjones.com. 10 Jan 2014
  17. 17.
    Dow Jones Indexes. http://www.djindexes.com. 5 Sept 2014
  18. 18.
    Durbin, J., Koopman, S.J.: Time Series Analysis by State Space Methods, vol. 38, 2nd edn. Oxford University Press, Oxford (2014)zbMATHGoogle Scholar
  19. 19.
    Eberhart, R., Kennedy, J.: A new optimizer using particle swarm theory. In: Proceedings of 6th International Symposium Micro Machine and Human Science (MHS), pp. 39–43 (1995)Google Scholar
  20. 20.
    Eberhart, R., Shi, Y.: Comparing inertia weights and constriction factors in particle swarm optimization. In: Proceedings of the IEEE Congress on Evolutionary Computation, vol. 1, pp. 84–88 (2000)Google Scholar
  21. 21.
    Engelbrech, P.: Fundamentals of Computational of Swarm Intelligence: Basic Particle Swarm Optimization, pp. 93–129. Wiley, New York (2005)Google Scholar
  22. 22.
    Erland, E., Ola, H.: Multivariate time series modeling, estimation and prediction of mortalities. Insur. Math. Econ. 65, 156–171 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Escalante, H.J., Montes, M., Sucar, L.E.: Particle swarm model selection. J. Mach. Learn. Res. 10, 405–440 (2009)Google Scholar
  24. 24.
    Gaxiola, F., Melin, P., Valdez, F., Castillo, O.: Interval type-2 fuzzy weight adjustment for backpropagation neural networks with application in time series prediction. Inf. Sci. 260, 1–14 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Gaxiola, F., Melin, P., Valdez, F., Castillo, O.: Optimization of type-2 fuzzy weight for neural network using genetic algorithm and particle swarm optimization. In: NaBIC, pp. 22–28 (2013)Google Scholar
  26. 26.
    Goldberg, D.E., Korb, B., Deb, K.: Messy genetic algorithms: motivation, analysis, and first results. Complex Syst. 3, 493–530 (1989)MathSciNetzbMATHGoogle Scholar
  27. 27.
    Goldberg, D.E.: Genetic Algorithms in Search Optimization and Machine Learning. Addison-Wesley, Reading (1989)zbMATHGoogle Scholar
  28. 28.
    Hagan, M.T., Demuth, H.B., Beale, M.H.: Neural Network Design. PWS Publishing, Boston (1996)Google Scholar
  29. 29.
    Hagras, H.: Comments on dynamical optimal training for interval type-2 fuzzy neural network (T2FNN). IEEE Trans. Syst. Man Cybern. B 36, 1206–1209 (2006)CrossRefGoogle Scholar
  30. 30.
    Haykin, S.: Adaptive Filter Theory. Prentice Hall, Englewood Cliffs (2002). ISBN 0-13-048434-2zbMATHGoogle Scholar
  31. 31.
    Historic Dow Jones Data, Yahoo Finance. http://finance.yahoo.com. 10 Jan 2014
  32. 32.
    Holland, J.: Adaptation in Natural and Artificial Systems. University of Michigan Press, Ann Arbor (1975)Google Scholar
  33. 33.
    Horikowa S., Furuhashi T., Uchikawa Y.: On fuzzy modeling using fuzzy neural networks with the backpropagation algorithm. IEEE Trans. Neural Netw. 3, 801–806 (1992)Google Scholar
  34. 34.
  35. 35.
  36. 36.
  37. 37.
  38. 38.
    Ishibuchi, H., Nozaki, K., Yamamoto, N., Tanaka, H.: Selecting fuzzy if-then rules for classification problems using genetic algorithms. IEEE Trans. Fuzzy Syst. 3, 260–270 (1995)CrossRefGoogle Scholar
  39. 39.
    Jang, J.S.R.: ANFIS: adaptive-network-based fuzzy inference systems. IEEE Trans. Syst. Man Cybern. 23, 665–685 (1992)CrossRefGoogle Scholar
  40. 40.
    Jang, J.S.R., Sun, C.T., Mizutani, E.: Neuro-Fuzzy and Soft Computing. Prentice-Hall, New York (1997)Google Scholar
  41. 41.
    Jang J.S.R.: Fuzzy modeling using generalized neural networks and Kalman filter algorithm. In: Proceedings of the Ninth National Conference on Artificial Intelligence.(AAAI-91), pp. 762–767 (1991)Google Scholar
  42. 42.
    Karnik, N.N., Mendel, J.M.: An Introduction to Type-2 Fuzzy Logic Systems. University of Southern California, Los Angeles (1998)CrossRefGoogle Scholar
  43. 43.
    Karnik, N.N., Mendel, J.M.: Applications of type-2 fuzzy logic systems to forecasting of time-series. Inf. Sci. 20, 89–111 (1999)CrossRefzbMATHGoogle Scholar
  44. 44.
    Karnik, N.N., Mendel, J.M., Qilian, L.: Type-2 fuzzy logic systems. IEEE Trans. Fuzzy Syst. 7, 643–658 (1999)CrossRefGoogle Scholar
  45. 45.
    Koza, J.R.: Genetic Programming. On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge (1992)zbMATHGoogle Scholar
  46. 46.
    Lee, C.H., Hong, J.L., Lin, Y.C., Lai, W.Y.: Type-2 fuzzy neural network systems and learning. Int. J. Comput. Cogn. 1, 79–90 (2003)Google Scholar
  47. 47.
    Lee, C.H., Lin, Y.C.: Type-2 fuzzy neuro system via input-to-state-stability approach. In: Liu, D., Fei, S., Hou, Z., Zhang, H., Sun, C. (eds.) International Symposium on Neural Networks. LNCS, vol. 4492, pp. 317–327. Springer, Heidelberg (2007)Google Scholar
  48. 48.
    Lee, H., Shin, G., Hong, S., Choi, J., Chun, M.: Post-chlorination process control based on flow prediction by time series neural network in water treatment plant. Int. J. Fuzzy Logic Intell. Syst. 16, 197–207 (2016)CrossRefGoogle Scholar
  49. 49.
    Lee, J., Lee, J.-H.: Constructing efficient regional hazardous weather prediction models through big data analysis. Int. J. Fuzzy Logic Intell. Syst. 16, 1–12 (2016)CrossRefGoogle Scholar
  50. 50.
    Lin, Y.C., Lee, C.H.: System identification and adaptive filter using a novel fuzzy neuro system. Int. J. Comput. Cogn. 5 1–12 (2007)Google Scholar
  51. 51.
    López, F., Santillán, R.J., Cruz, S.: Volatility dependence structure between the Mexican stock exchange and the world capital market. Investig. Econ. 74, 69–97 (2015)CrossRefGoogle Scholar
  52. 52.
    Mackey, M.C., Glass, L.: Oscillation and chaos in physiological control systems. Science 197, 287–289 (1997)CrossRefGoogle Scholar
  53. 53.
    Mamdani, E.H., Assilian, S.: An experiment in linguistic synthesis with a fuzzy logic controller. Int. J. Man-Mach. Stud. 7, 1–13 (1975)CrossRefzbMATHGoogle Scholar
  54. 54.
    Melin, P., Soto, J., Castillo, O., Soria, J.: A new approach for time series prediction using ensembles of ANFIS models. Exp. Syst. Appl. 39, 3494–3506 (2012)CrossRefGoogle Scholar
  55. 55.
    Mendel, J.M.: Uncertain Rule-Based Fuzzy Logic Systems: Introduction and New Directions, pp. 259–674. Prentice-Hall, Englewood Cliffs (2001)zbMATHGoogle Scholar
  56. 56.
    Pagano, M.S., Peng, L., Schwartz, R.A.: A call auction’s impact on price formation and order routing: evidence from the NASDAQ stock market. J. Financ. Mark. 16, 331–361 (2013)CrossRefGoogle Scholar
  57. 57.
    Pedrycz, W.: Concepts and design aspects of granular models of type-1 and type-2. Int. J. Fuzzy Logic Intell. Syst. 15, 87–95 (2015)CrossRefGoogle Scholar
  58. 58.
    Pedrycz, W.: Fuzzy Evolutionary Computation. Kluwer Academic Publishers, Dordrecht (1997)CrossRefzbMATHGoogle Scholar
  59. 59.
    Pedrycz, W.: Fuzzy Modelling: Paradigms and Practice. Kluwer Academic Press, Dordrecht (1996)CrossRefzbMATHGoogle Scholar
  60. 60.
    Pulido, M., Melin, P.: A New Method for Type-2 Fuzzy Integration in Ensemble Neural Networks Based on Genetic Algorithms. In: Castillo, O., et al. (eds.) Recent Advances on Hybrid Intelligent Systems, vol. 451, pp. 173–182. Springer, New York (2013)CrossRefGoogle Scholar
  61. 61.
    Pulido, M., Melin, P., Castillo, O.: Particle swarm optimization of ensemble neural networks with fuzzy aggregation for time series prediction of the mexican stock exchange. Inf. Sci. 280, 188–204 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  62. 62.
    Pulido, M., Melin, P.: Optimization of ensemble neural networks with fuzzy integration using the particle swarm algorithm for time series prediction. In: Melin, P., et al. (eds.) Design of Intelligent Systems Based on Fuzzy Logic, Neural Networks and Nature-Inspired Optimization, vol. 601, pp. 171–184. Springer, New York (2015)CrossRefGoogle Scholar
  63. 63.
    Russell, S., Norvig, P.: Artificial Intelligence: A Modern Approach. Prentice-Hall, Englewood Cliffs (2003)zbMATHGoogle Scholar
  64. 64.
    Shu-Xian, L., Xian-Shuang, Y., Hong-Yun, Q., Hai-Feng, H.: A novel model of leaky integrator echo state network for time-series prediction. Neurocomputing 159, 58–66 (2015)CrossRefGoogle Scholar
  65. 65.
    Sidaoui, J.: The Mexican financial system: reforms and evolution 1995–2005. BIS Pap. 28, 277–293 (2006)Google Scholar
  66. 66.
    Soto, J., Melin, P., Castillo, O.: Time series prediction using ensembles of ANFIS models with genetic optimization of interval type-2 and type-1 fuzzy integrators. Int. J. Hybrid Intell. Syst. 11, 211–226 (2014)CrossRefGoogle Scholar
  67. 67.
    Takagi, T., Sugeno, M.: Derivation of fuzzy control rules from human operation control actions. In: Proceedings of the IFAC Symposium on Fuzzy Information, Knowledge Representation and Decision Analysis, pp. 55–60 (1983)Google Scholar
  68. 68.
    Takagi, T., Sugeno, M.: Fuzzy identification of systems and its applications to modeling and control. IEEE Trans. Syst. Man Cybern. 15, 116–132 (1985)CrossRefzbMATHGoogle Scholar
  69. 69.
    Valipour, M.: Ability of Box–Jenkins models to estimate of reference potential evapotranspiration. A case study Mehrabad synoptic station, Tehran, Iran. IOSR J. Agric. Vet. Sci. 1, 1–11 (2012)CrossRefGoogle Scholar
  70. 70.
    Valipour, M.: Analysis of potential evapotranspiration using limited weather data. Appl. Water Sci. 7, 187–197 (2017)CrossRefGoogle Scholar
  71. 71.
    Valipour, M.: How much meteorological information is necessary to achieve reliable accuracy for rainfall estimations? Agriculture 6, 1–9 (2016)CrossRefGoogle Scholar
  72. 72.
    Valipour, M., Mohammad, A.G.S., Mahmoud, R.-S.: Selecting the best model to estimate potential evapotranspiration with respect to climate change and magnitudes of extreme events. Agric. Water Manag. 180, 50–60 (2017)CrossRefGoogle Scholar
  73. 73.
    Valipour, M., Mohammad, E.B., Seyyed, M.R.B.: Comparison of the ARMA, ARIMA, and the autoregressive artificial neural network models in forecasting the monthly inflow of Dez dam reservoir. J. Hydrol. 476, 433–441 (2013)CrossRefGoogle Scholar
  74. 74.
    Wang, C.H., Cheng, C.S., Lee, T.T.: Dynamical optimal training for interval type-2 fuzzy neural network (T2FNN). IEEE Trans. Syst. Man Cybern. B Cybern. 34, 1462–1477 (2004)CrossRefGoogle Scholar
  75. 75.
    Wei, W.W.S.: Time Series Analysis: Univariate and Multivariate Methods, vol. 1, pp. 40–100. Addison-Wesley, Reading (2003)Google Scholar
  76. 76.
    Weina, W., Witold, P., Xiaodong, L.: Time series long-term forecasting model based on information granules and fuzzy clustering. Eng. Appl. Artif. Intell. 41, 17–24 (2015)CrossRefGoogle Scholar
  77. 77.
    Wu, D., Mendel, J.M.: A vector similarity measure for interval type-2 fuzzy sets and type-1 fuzzy sets. Inf. Sci. 178, 381–402 (2008)CrossRefzbMATHGoogle Scholar
  78. 78.
    Wu, D., Wan-Tan, W.: Genetic learning and performance evaluation of interval type-2 fuzzy logic controllers. Eng. Appl. Artif. Intell. 19, 829–841 (2006)CrossRefGoogle Scholar
  79. 79.
    Zadeh, L.A.: Fuzzy logic = computing with words. IEEE Trans. Fuzzy Syst. 4, 103–111 (1996)CrossRefGoogle Scholar
  80. 80.
    Zadeh, L.A.: Fuzzy logic. Computer 1, 83–93 (1988)CrossRefGoogle Scholar
  81. 81.
    Zadeh, L.A.: Fuzzy logic, neural networks and soft computing. Commun. ACM 37, 77–84 (1994)CrossRefGoogle Scholar

Copyright information

© Taiwan Fuzzy Systems Association and Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Tijuana Institute of TechnologyTijuanaMexico

Personalised recommendations