Skip to main content
SpringerLink
Log in

Comparison of Genetic Algorithm and Particle Swarm Optimization of Ensemble Neural Networks for Complex Time Series Prediction

  • Chapter
  • First Online:
Recent Advances of Hybrid Intelligent Systems Based on Soft Computing

Part of the book series: Studies in Computational Intelligence ((SCI,volume 915))

Abstract

In this paper the development of a method for the design of the general architecture of ensemble neural networks for time series prediction is presented. This method consists on the optimization of the ensemble neural network with response integration based on type-1 and type-2 fuzzy logic. The optimization algorithms that are used to illustrate the proposed method are the genetic algorithm and the particle swarm optimization algorithm and the purpose of applying these methods is to find the best possible architecture of the neural network ensemble with the lowest prediction error. Simulation results with several complex times series are presented to show the advantage of the proposed method, also comparisons are made with other time series, such as Mexican Stock Exchange, and Dow Jones Stock Exchange.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. P.D. Brockwell, A.D. Richard, Introduction to Time Series and Forecasting (Springer, New York, 2002), pp. 1–219

    Book  Google Scholar 

  2. O. Castillo, P. Melin, Hybrid intelligent systems for time series prediction using neural networks, fuzzy logic, and fractal theory. Neural Netw. IEEE Trans. 13(6), 1395–1408 (2002)

    Article  Google Scholar 

  3. O. Castillo, P. Melin, Simulation and forecasting complex economic time series using neural networks and fuzzy logic. Proc. Int. Neural Netw. 3, 1805–1810 (2001)

    Google Scholar 

  4. O. Castillo, P. Melin, Simulation and forecasting complex financial time series sing neural networks and fuzzy logic. Proc. IEEE Int. Conf. Syst. Man Cybern. 4, 2664–2669 (2001)

    Article  Google Scholar 

  5. Cherkauer K.J., Human expert level performance on a scientific image analysis task by a system using combined artificial neural networks, in Proceedings of AAAI-96Workshop on Integrating Multiple Learned Models for Improving and Scaling Machine Learning Algorithms ed by P. Chan, S. Stolfo, D. Wolpert (AAAI Press, Portland, OR, Menlo Park, CA, 1996), pp. 15–21

    Google Scholar 

  6. P. Cunningham, J. Carney, S. Jacob, Stability problems with artificial neural networks and the ensemble solution. Artif. Intell. Med. 20(3), 217–225 (2000)

    Article  Google Scholar 

  7. N. Davey, S. Hunt, R. Frank, Time Series Prediction and Neural Networks (University of Hertfordshire, Hatfield, UK, 1999)

    MATH  Google Scholar 

  8. M. Dorigo, T. Stutzle, Ant Colony Optimization “Ant Colony Optimization Theory” (A Bradford Book The Milt Press, London England, Massachusetts Institute of Technology, 2004) pp. 121–151

    Google Scholar 

  9. H. Drucker, R. Schapire, P. Simard, Improving performance in neural networks using a boosting algorithm, in Advances in Neural Information Processing Systems, vol. 5, ed. by S.J. Hanson, J.D. Cowan, C.L. Giles (Morgan Kaufmann, Denver, CO, San Mateo, CA, 1993), pp. 42–49

    Google Scholar 

  10. Dow Jones Company. http://www.dowjones.com. 10 Jan 2010

  11. Dow Jones Indexes. http://www.djindexes.com. 5 Sep 2010

  12. Engelbrech. P., in Fundamentals of Computational of Swarm Intelligence, Basic Particle Swarm Optimization, (Wiley y Sons, 2005), pp. 93–129

    Google Scholar 

  13. R.R. Yager et al., in Fuzzy Sets and Applications: Selected Papers, ed. by L.A. Zadeh (John Wiley, New York, 1987)

    Google Scholar 

  14. D. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning. Ed. (Addison Wesley, 1989)

    Google Scholar 

  15. S. Gutta, H. Wechsler, Face recognition using hybrid classifier systems, in Proceedings of ICNN-96 (IEEE Computer Society Press, Washington, DC, Los Alamitos, CA, 1996), pp. 1017–1022

    Google Scholar 

  16. J. Hampshire, A. Waibel, A novel objective function for improved phoneme recognition using time- delay neural networks. IEEE Trans. Neural Netw. 1(2), 216–228 (1990)

    Article  Google Scholar 

  17. L.K. Hansen, P. Salomon, Neural network ensembles. IEEE Trans. Pattern Anal. Mach. Intell. 12(10), 993–1001 (1990)

    Article  Google Scholar 

  18. L.K. Hansen, P. Salomon, Ensemble methods for hand written digit recognition, in Proceedings of IEEE Workshop on Neural Networks for Signal Processing (IEEE Press, Helsingoer, Denmark, Piscataway, NJ, 199)2, pp. 333–342

    Google Scholar 

  19. J. Holland, Adaptation in Natural and Artificial Systems (University of Michigan Press, 1975)

    Google Scholar 

  20. F.J. Huang, Z. Huang. H.-J. Zhang, T.H. Chen, Pose invariant face recognition, in Proceedings of 4th IEEE International Conference on Automatic Face and Gesture Recognition (IEEE Computer Society Press, Grenoble, France, Los Alamitos, CA, 2000), pp. 245–250

    Google Scholar 

  21. N. Karnik, J.M. Mendel, Applications of type-2 fuzzy logic systems to forecasting of time-series. 120(1–4), 89–111 (1999)

    Google Scholar 

  22. A. Kehagias, V. Petridis, Predictive modular neural networks for time series classification. Neural Netw. 10(1), 31–49 (1997)

    Article  Google Scholar 

  23. A. Krogh, J. Vedels, Neural network ensembles, cross validation, and active learning, in Advances in Neural Information Processing Systems 7, ed. by G. Tesauro, D. Touretzky, T. Leen (MIT Press, CO, Denver, Cambridge, MA, 1995), pp. 231–238

    Google Scholar 

  24. M.C. Mackey, Adventures in Poland: having fun and doing research with Andrzej Lasota. Mat. Stosow, 5–32 (2007)

    Google Scholar 

  25. M.C. Mackey, L. Glass, Oscillation and chaos in physiological control systems. Science 197, 287–289 (1997)

    Article  Google Scholar 

  26. Maclin R., Shavlik J.W., Combining the predictions of multiple classifiers: using competitive learning to initialize neural networks, in Proceedings of IJCAI-95 (Montreal, Canada, Morgan Kaufmann, San Mateo, CA, 1995), pp. 524–530

    Google Scholar 

  27. K. Man, K. Tang, S. Kwong, in Genetic Algorithms and Designs, “Introduction, Background and Biological Background Ed. (Springer, London Limited, 1998), pp. 1–62

    Google Scholar 

  28. L.P. Maguire, B. Roche, T.M. McGinnity, J. McDaidL, Predicting a chaotic time series using a fuzzy neural network. Inf. Sci. 112(1–4), 125–136 (1998)

    Article  Google Scholar 

  29. J. Mao, A case study on bagging, boosting and basic ensembles of neural networks for OCR, in Proceedings of IJCNN-98, vol. 3. (IEEE Computer Society Press, Anchorage, AK, Los Alamitos, CA, 1998), pp. 1828–1833

    Google Scholar 

  30. MATLAB 2007, Historical Mackey Glass data

    Google Scholar 

  31. Mexico Bank Database, http://www.banxico.org.mx. 30 Aug 2011

  32. P. Melin, O. Castillo, S. Gonzalez, J. Cota, W. Trujillo, P. Osuna, Design of Modular Neural Networks with Fuzzy Integration Applied to Time Series Prediction, vol. 41 (Springer, Berlin/Heidelberg, 2007), pp. 265–273

    Google Scholar 

  33. I.M. Multaba, M.A. Hussain, Application of Neural Networks and Other Learning. Technologies in Process Engineering (Imperial Collage Press, London, 2001)

    Book  Google Scholar 

  34. J.S.R. Jang, C.T. Sun, E. Mizutani, Neuro-Fuzzy and Soft Computing (Prentice Hall, Englewood Cliffs NJ, 1996)

    Google Scholar 

  35. D.W. Opitz, J.W. Shavlik, Generating accurate and diverse members of a neural network ensemble, in Advances in Neural Information Processing Systems 8, ed. by D.S. Touretzky, M.C. Mozer, M.E. Hasselmo (MIT Press, Cambridge, MA, Denver, CO, 1996), pp. 535–541

    Google Scholar 

  36. E.A. Plummer, in Time Series Forecasting With Feed-Forward Neural Networks: Guidelines and Limitations. University of Wyoming (2000)

    Google Scholar 

  37. C. Reeves, J. Row, Genetic Algorithms: Principles and Perspectives, “Optimization” (Kluwer Academic Publishers, New York, Boston, Dordrecht, London, 2002), pp. 4–8

    Book  Google Scholar 

  38. A. Sharkey, One combining Artificial of Neural Nets (Department of Computer Science University of Sheffield, U.K., 1996)

    Google Scholar 

  39. A. Sharkey, Combining Artificial Neural Nets: Ensemble and Modular Multi-Net Systems (Springer, London, 1999)

    Book  Google Scholar 

  40. P. Sollich, A. Krogh, Learning with ensembles: how over-fitting can be useful, in Advances in Neural Information Processing Systems, vol. 8, ed. by D.S. Touretzky, M.C. Mozer, M.E. Hasselmo (MIT Press, Cambridge, MA, Denver, CO, 1996), pp. 190–196

    Google Scholar 

  41. S. Soltani, On the use of the wavelet decomposition for time series prediction. Neurocomputing 48(1–4), 267–277 (2002)

    Article  Google Scholar 

  42. M. Štěpnička, P. Cortez, J.P. Donate, L. Štěpničková, Forecasting seasonal time series with computational intelligence: On recent methods and the potential of their combinations. Expert Syst. Appl. 40(6), 1981–1992 (2013)

    Article  Google Scholar 

  43. W.W.S. Wei, Time Series Analysis: Univariate and Multivariate Methods, vol. 1 (Addison-Wesley, Reading, MA, 1994), pp. 40–100

    Google Scholar 

  44. R.N. Yadav, P.K. Kalra, J. John, Time series prediction with single multiplicative neuron model. Soft Comput. Time Ser. Prediction Appl. Soft Comput. 7(4), 1157–1163 (2007)

    Google Scholar 

  45. X. Yao, Y. Liu, Making use of population in formation in evolutionary artificial neural networks. IEEE Trans. Syst. Man Cybern. Part B Cybern. 28(3), 417–425 (1998)

    Google Scholar 

  46. J. Yen, R. Langari, Fuzzy Logic: Intelligence, Control and Information Ed. (Prentice Hall, Englewood Cliffs NJ, 1999)

    Google Scholar 

  47. L. Zhao, Y. Yang, PSO-based single multiplicative neuron model for time series prediction. Expert Syst. Appl. 36(2), 2805–2812 (2009). (Part 2)

    Google Scholar 

  48. Z.-H. Zhou, Y. Jiang, Y.-B. Yang, S.-F. Chen, Lung cancer cell identification based on artificial neural network ensembles. Artif. Intell. Med. 24(1), 25–36 (2002)

    Article  Google Scholar 

  49. M.A. Sanchez, O. Castillo, J.R. Castro, P. Melin, Fuzzy granular gravitational clustering algorithm for multivariate data. Inf. Sci. 279, 498–511 (2014)

    Article  MathSciNet  Google Scholar 

  50. D. Sánchez, P. Melin, Optimization of modular granular neural networks using hierarchical genetic algorithms for human recognition using the ear biometric measure. Eng. Appl. Artif. Intell. 27, 41–56 (2014)

    Article  Google Scholar 

  51. M.A. Sanchez, J.R. Castro, O. Castillo, O. Mendoza, A. Rodriguez-Diaz, P. Melin, Fuzzy higher type information granules from an uncertainty measurement. Granular Comput. 2(2), 95–103 (2017)

    Article  Google Scholar 

  52. F. Gaxiola, P. Melin, F. Valdez, J.R. Castro, O. Castillo, Optimization of type-2 fuzzy weights in backpropagation learning for neural networks using GAs and PSO. Appl. Soft Comput. 38, 860–871 (2016)

    Article  Google Scholar 

Download references

Acknowledgements

We would like to express our gratitude to the CONACYT, Tijuana Institute of Technology for the facilities and resources granted for the development of this research.

Author information

Authors and Affiliations

  1. Tijuana Institute of Technology, Tijuana, Mexico

    Martha Pulido & Patricia Melin

Authors
  1. Martha Pulido
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Patricia Melin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Martha Pulido .

Editor information

Editors and Affiliations

  1. Division of Graduate Studies and Research, Tijuana Institute of Technology, Tijuana, Mexico

    Patricia Melin

  2. Division of Graduate Studies and Research, Tijuana Institute of Technology, Tijuana, Mexico

    Oscar Castillo

  3. Systems Research Institute, Polish Academy of Sciences, Warsaw, Poland

    Janusz Kacprzyk

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Pulido, M., Melin, P. (2021). Comparison of Genetic Algorithm and Particle Swarm Optimization of Ensemble Neural Networks for Complex Time Series Prediction. In: Melin, P., Castillo, O., Kacprzyk, J. (eds) Recent Advances of Hybrid Intelligent Systems Based on Soft Computing. Studies in Computational Intelligence, vol 915. Springer, Cham. https://doi.org/10.1007/978-3-030-58728-4_3

Download citation

Publish with us

Policies and ethics

Search

Navigation