Evolving Systems

, Volume 4, Issue 2, pp 99–117 | Cite as

Evolving integrated multi-model framework for on line multiple time series prediction

  • Russel Pears
  • Harya Widiputra
  • Nikola Kasabov
Original Paper


Time series prediction has been extensively researched in both the statistical and computational intelligence literature with robust methods being developed that can be applied across any given application domain. A much less researched problem is multiple time series prediction where the objective is to simultaneously forecast the values of multiple variables which interact with each other in time varying amounts continuously over time. In this paper we describe the use of a novel integrated multi-model framework (IMMF) that combined models developed at three different levels of data granularity, namely the global, local and transductive models to perform multiple time series prediction. The IMMF is implemented by training a neural network to assign relative weights to predictions from the models at the three different levels of data granularity. Our experimental results indicate that IMMF significantly outperforms well established methods of time series prediction when applied to the multiple time series prediction problem.


DENFIS Local trend models Dynamic interaction networks Integrated multi model framework Transductive modelling 



The work was funded by the Knowledge Engineering and Discovery Research Institute (KEDRI and the School of Computing and Mathematical Sciences of the Auckland University of Technology, New Zealand.


  1. Angelov P.A., Filev D.P. (2004) An Approach to Online Identification of Takagi–Sugeno Fuzzy Models IEEE Transactions on Systems. Man and Cybernetics - Part B 34(1):484–498CrossRefGoogle Scholar
  2. Angelov PA, Filev D (2005) Simpl_eTS: a simplified method for learning evolving Takagi Sugeno fuzzy models. In: Proceedings of the 14th IEEE international conference on fuzzy systems, pp 1068–1073Google Scholar
  3. Angelov P, Filev DP, Kasabov N (2010) Evolving intelligent systems : methodology and applications. In: Angelov P, Filev DP, Kasabov N (eds) IEEE press series on computational Intelligence, Wiley, New YorkGoogle Scholar
  4. Antoniou A., Pescetto G., Violaris A.(2003) Modelling international price relationships and interdependencies between the stock index and stock index futures markets of three eu countries: A multivariate analysis. Journal of Business Finance & Accounting. 30:645–677CrossRefGoogle Scholar
  5. Bierens H (1994) The Nadaraya–Watson kernel regression function estimator. Technical Report, NATO ASI SeriesGoogle Scholar
  6. Cevikalp H., Polikar R. (2008) Local classifier weighting by quadratic programming. IEEE Transactions on Neural Networks 19, 1832–1838CrossRefGoogle Scholar
  7. Chiang T., Doong S. (2001) Empirical analysis of stock returns and volatility: Evidence from seven asian stock markets based on TARGARCH model. Rev Quant Financ Acc 17(3):301–318CrossRefGoogle Scholar
  8. Chowdhury A.(1994) Stock market interdependencies: Evidence from the asian NIEs. J Macroecon 16(4):629–651CrossRefGoogle Scholar
  9. Collins D., Biekpe N.(2003)Contagion and interdependence in african stock markets. South African Journal of Economics 71(1):181–194CrossRefGoogle Scholar
  10. Davidson EH (2006) The regulatory genome: gene regulatory networks in development and evolution. Academic Press, Burlington.
  11. Dovzan D, Logar V, Skrjanc I (2012) Solving the sales prediction problem with fuzzy evolving methods. In: IEEE congress on evolutionary computation, pp 1–8Google Scholar
  12. Hai-Jun R, Sundararajan N, Guang-Bin H, Saratchandran P (2006) Sequential Adaptive Fuzzy Inference System (SAFIS) for nonlinear system identification and prediction. Journal of Fuzzy Sets and Systems 157(9):1260–1275zbMATHCrossRefGoogle Scholar
  13. Islam M., Yao X., Nirjon S., Islam M., Murase K.(2008) Bagging and Boosting Negatively Correlated Neural Networks. IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS PART B: CYBERNETICS 38(3):771–784CrossRefGoogle Scholar
  14. Joachims T (1999) Transductive inference for text classification using support vector machines. In: Proceedings of the sixteenth international conference on machine learning, ICMLGoogle Scholar
  15. Kasabov N. (2001) Evolving fuzzy neural networks for supervised/unsupervised on-line knowledge-based learning. IEEE Transactions on Systems, Man and Cybernetics Part B 31:902–918CrossRefGoogle Scholar
  16. Kasabov N. (2007a) Global, local and personalised modeling and pattern discovery in bioinformatics: An integrated approach. Pattern Recognition Letters 28(6):673–685CrossRefGoogle Scholar
  17. Kasabov N (2007b) Evolving connectionist systems: the knowledge engineering approach, 2nd edn. Springer Verlag, LondonGoogle Scholar
  18. Kasabov N, Pang S (2003) Transductive support vector machines and applications in bioinformatics for promoter recognition. In: Proceedings of the 2003 international conference on neural networks and signal processingGoogle Scholar
  19. Kasabov N, Song Q (2002) DENFIS: dynamic evolving neural-fuzzy inference system and its application for time-series prediction. IEEE Transactions on Fuzzy Systems 10(2):144–154CrossRefGoogle Scholar
  20. Kasabov N, Chan Z, Jain V, Sidorov I, Dimitrov D (2004) Gene regulatory network discovery from time-series gene expression data: a computational intelligence approach. Lecture Notes in Computer Science 3316:1333–1353Google Scholar
  21. Kasbov N, Hamed H (2011) Quantum-inspired Particle Swarm Optimisation for Integrated Feature and Parameter Optimisation of Evolving Spiking Neural Networks. International Journal of Artificial Intelligence 7(11):114–124Google Scholar
  22. Kim R, Ji H, Wong W (2006) An improved distance measure between the expression profiles linking co-expression and co-regulation in mouse. BMC Bioinformatics 7:1–8CrossRefGoogle Scholar
  23. Levine M, Davidson EH (2005) Gene regulatory networks for development. In: Gene regulatory networks for development proceedings of the national academy of sciences of the United States of America, vol 102(14), pp 4936–4942. doi: 10.1073/pnas.0408031102
  24. Li F, Wechsle H (2004) Watch list face surveillance using transductive inference. In: ICBA’04Google Scholar
  25. Li X, Chen H, Li J, Zhang Z (2010) Gene function prediction with gene interaction networks: a context graph kernel approach. Trans. Info. Tech. Biomed 14(1):119–128CrossRefGoogle Scholar
  26. Lucey B, Muckley C (2010) Global stock market interdependencies and long-term portfolio diversification. Technical Report, SSRN eLibraryGoogle Scholar
  27. Lughofer E (2008) FLEXFIS: A Robust Incremental Learning Approach for Evolving Takagi Sugeno Fuzzy Models IEEE Transactions on Fuzzy Systems. 16(6):1393–1410CrossRefGoogle Scholar
  28. Lughofer E (2011) Evolving fuzzy systems—methodologies, advanced concepts and applications. Springer, BerlinGoogle Scholar
  29. Masin R, Masih A. (2001) Dynamic modeling of stock market interdependencies: An empirical investigation of australia and the asian NICs. Review of Pacific Basin Financial Markets and Policies 4(2):235–264CrossRefGoogle Scholar
  30. Ozawa S, Pang S, Kasabov N (2010) Online feature extraction for evolving intelligent systems. In: Angelov P, Filev DP, Kasabov N (eds) Evolving intelligent systems: methodology and applications, John Wiley & Sons, Inc., Hoboken. doi: 10.1002/9780470569962.ch7
  31. Phylaktis K, Ravazzolo F (2005) Stock market linkages in emerging markets: implications for international portfolio diversification. Journal of International Financial Markets 15(2):91–106CrossRefGoogle Scholar
  32. Rodrigues P, Gama J, Pedroso J (2008) Hierarchical clustering of time-series data streams. IEEE Trans. on Knowl. and Data Eng. 20:615–627CrossRefGoogle Scholar
  33. Shaker A, Hullermeier E (2012) Instance-based classification and regression on data streams. In: Sayed-Mouchaweh M, Lughofer E (eds) Learning in non-stationary environments. Springer Verlag, pp 185–201Google Scholar
  34. Song Q, Kasabov N (2001) ECM—a novel on-line, evolving clustering method and its applications. In: Posner MI (ed) Foundations of cognitive scienceGoogle Scholar
  35. Song Q, Kasabov N (2005) NFI: a neuro-fuzzy inference method for transductive reasoning. IEEE Transactions on Fuzzy Systems 13(6):799–808CrossRefGoogle Scholar
  36. Takagi T, Sugano M (1985) Fuzzy identification of systems and its applications to modeling and control. IEEE Transactions on Systems, Man, and Cybernetics 15(1):116–132zbMATHCrossRefGoogle Scholar
  37. Vitousek P (1992) Global environmental change: An introduction. Review of Ecology and Systematics 23:1–14CrossRefGoogle Scholar
  38. Watts M (2008) A decade of Kasabov’s evolving connectionist systems: a review. IEEE Transactions on Systems, Man, and Cybernetics. Part C 39(3):253–269CrossRefGoogle Scholar
  39. Welling M (2001) The Kalman filter. Technical Report, California Institute of TechnologyGoogle Scholar
  40. Widiputra H (2011) Integrated multi-model framework for adaptive multiple time-series analysis and modelling. Ph.D. thesis, Knowledge Engineering and Discovery Research Institute, Auckland University of TechnologyGoogle Scholar
  41. Widiputra H, Pears R, Kasabov N (2008) Personalised modelling for multiple time-series data prediction: a preliminary investigation in asia pacific stock market indexes movement. In: Proceedings of the 15th international conference on advances in neuro-information processing, 2008, ICONIP ’08, 1237–1244Google Scholar
  42. Widiputra H, Kho H, Lukas, Pears R, Kasabov N (2009) A novel evolving clustering algorithm with polynomial regression for chaotic time-series prediction. In: Proceedings of the 16th international conference on neural information processing: part II series, ICONIPGoogle Scholar
  43. Widiputra H, Pears R, Serguieva A, Kasabov N (2009) Dynamic interaction networks in modelling and predicting the behaviour of multiple interactive stock markets. Intelligent Systems in Accounting, Finance and Management 16:189–205CrossRefGoogle Scholar
  44. Widiputra H, Pears R, Kasabov N (2011a) Dynamic Interaction Networks versus Local Trend Models for Multiple Time Series Prediction. Cybernetics and Systems 42(2):100–123CrossRefGoogle Scholar
  45. Widiputra H, Pears R, Kasabov N (2011b) Multiple time-series prediction through multiple time-series relationships profiling and clustered recurring trends. In: Proceedings of the Pacific Asia conference on knowledge discovery and data mining 2011, PAKDD 11Google Scholar
  46. Widrow B, Lehr M (1993) Adaptive neural networks and their applications. International Journal of Intelligent Systems 8:453–507zbMATHCrossRefGoogle Scholar
  47. Wu D, Bennett K, Cristianini N, Shawe-Taylor J (1999a) Large margin trees for induction and transduction. In: Proceedings of the sixteenth international conference on machine learning, ICML 99Google Scholar
  48. Wu D, Bennett KP, Cristianini N, Shawe-taylor J, Holloway R (1999b) Functional multi-layer perceptron: a non-linear tool for functional data analysis. In: Proceedings of the sixteenth international conference on machine learning (ICML 1999), Bled, Slovenia, June 27–30, 1999Google Scholar
  49. Zadeh L (1973) Outline of a new approach to the analysis of complex systems and decision processes. IEEE Transactions on Systems. Man and Cybernetics Part C 3(1):28–44MathSciNetzbMATHGoogle Scholar
  50. Zhou Z, Jiang Y (2003) Medical diagnosis with C4.5 rule preceded by artificial neural network ensemble. IEEE Transactions on Information Technology in Biomedicine 7:37–42CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  1. 1.School of Computing and Mathematical Sciences, Auckland University of TechnologyAucklandNew Zealand
  2. 2.Knowledge Engineering and Discovery Research Institute, Auckland University of TechnologyAucklandNew Zealand

Personalised recommendations