Skip to main content
SpringerLink
Log in

Extracting online information from dual and multiple data streams

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In this paper, we consider the challenging problem of finding shared information in multiple data streams simultaneously. The standard statistical method for doing this is the well-known canonical correlation analysis (CCA) approach. We begin by developing an online version of the CCA and apply it to reservoirs of an echo state network in order to capture shared temporal information in two data streams. We further develop the proposed method by forcing it to ignore shared information that is created from static values using derivative information. We finally develop a novel multi-set CCA method which can identify shared information in more than two data streams simultaneously. The comparative effectiveness of the proposed methods is illustrated using artificial and real benchmark datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Akaho S, kernal A (2006) method for canonical correlation analysis. In: Proceedings of the international meeting of the Psychometric Society (IMPS2001), Springer (arXiv preprint cs/0609071)

  2. BieBmann F, Meinecke FC, Gretton A, Rauch A, Rainer G, Logothetis NK, Muller KR (2010) Temporal kernal CCA and its application in multimodal neuronal data analysis. Mach Learn 79(1–2):5–27

    Article  MathSciNet  Google Scholar 

  3. Cambria E, Fu J, Bisio F, Poria S (2015) AffectiveSpace2: enabling affective intuition for concept-level sentiment analysis. In: AAAI, pp 508–514

  4. Cambria E, Gastaldo P, Bisio F, Zunino R (2015) An ELM-based model for affective analogical reasoning. Neurocomputing 149(Part A):443–455

  5. Friman O, Borga M, Lundberg P, Knutsson H (2002) Exploratory fmri analysis by autocorrelation maximization. Neuroimage 16(2):454–464

    Article  Google Scholar 

  6. Gastaldo P, Zunino R, Cambria E, Decherchi S (2013) Combining elm with random projections. IEEE Intell Syst 28(6):46–48

    Google Scholar 

  7. Gou Z, Fyfe C (2001) A family of networks which perform canonical correlation analysis. Int J Knowl Based Intell Eng Syst 5(2):76–82

    Google Scholar 

  8. Gou Z, Fyfe C (2003) A canonical correlation neural network for multicollinearity and functional data. Neural Netw 17(2):285–293

    Article  MATH  Google Scholar 

  9. Gros C (2009) Cognitive computation with autonomously active neural networks: an emerging field. Cognit Comput 1(1):77

    Article  Google Scholar 

  10. Hardoon DR, Szedmak S, Shawe-Taylor J (2004) Canonical correlation analysis: an overview with application to learning methods. Neural Comput 16(12):2639–2664

    Article  MATH  Google Scholar 

  11. Hotelling H (1936) Relations between two sets of variates. Biometrika 28(3/4):321–377

    Article  MATH  Google Scholar 

  12. Huang GB (2014) An insight into extreme learning machines: random neurons, random features and kernels. Cognit Comput 6(3):376

    Article  Google Scholar 

  13. Kettenring JR (1971) Canonical analysis of several sets of variables. Biometrika 58(3):433

    Article  MathSciNet  MATH  Google Scholar 

  14. Lai PL, Fyfe C (1999) A neural implementation of canonical correlation analysis. Neural Netw 12(10):1391–1397

    Article  Google Scholar 

  15. Lai PL, Fyfe C (2001) Kernel and nonlinear canonical correlation analysis. Int J Neural Syst 10(5):365–377

    Article  Google Scholar 

  16. LeCun Y, Cortes C (1998) The MNIST database of handwritten digits. The dataset is available at http://yann.lecun.com/exdb/mnist

  17. Lemire D (2009) Faster retrieval with a two-pass dynamic-time-warping lower bound. Pattern Recognit 42(9):2169

    Article  MATH  Google Scholar 

  18. Lukosevicius M, Jaeger H (2009) Reservoir computing approaches to recurrent neural network training. Comput Sci Rev 3(3):127–149

    Article  MATH  Google Scholar 

  19. Malik ZK, Hussain A, Wu J (2014) Novel biologically inspired approaches to extracting online information from temporal data. Cognit Comput 6(3):1–13

    Article  Google Scholar 

  20. Malik ZK, Hussain A, Wu J (2016) An online generalized eigenvalue version of Laplacian eigenmap for visual big data. Neurocomputing 173(2):127–136

    Article  Google Scholar 

  21. Malik ZK, Hussain A, Wu QJ (2016) Multilayered echo state machine: a novel architecture and algorithm. IEEE Trans Cybern PP(99):1–14

  22. Mardia KV, Kent JT, Bibby JM (1980) Multivariate analysis. Academic Press, Cambridge

  23. Nielsen AA (2002) Multi-set canonical correlations analysis and multispectral, truely multitemporal remote sensing data. IEEE Trans Image Process 11(3):293

    Article  Google Scholar 

  24. Poria S, Gelbukh A, Cambria E, Hussain A, Huang GB (2014) Emosenticspace: a novel framework for affective common-sense reasoning. Knowl Based Syst 69:108–123. doi:10.1016/j.knosys.2014.06.011

    Article  Google Scholar 

  25. Schrauwen B, Verstraeten D, Van Campenhout J (2007) An overview of reservoir computing: theory, application and implementation. In: Proceedings of the 15th European symposium on artificial neural networks, pp 471–482

  26. Steil JJ (2007) Online reservoir adaptation by intrinsic plasticity for backpropagation-decorrelation and echo state learning. Neural Netw 20(3):353–364

    Article  MATH  Google Scholar 

  27. Via J, Santamaria I, Perez J (2007) A learning algorithm for adaptive canonical correlation analysis of several data sets. Neural Netw 20(1):139–152

    Article  MATH  Google Scholar 

  28. Wang X, Crowe M, Fyfe C (2012) Dual stream data exploration. Int J Data Min Model Manag 4(2):188–202

    Article  Google Scholar 

  29. Wiskott L, Sejnowski TJ (2002) Slow feature analysis: unsupervised learning of invariances. Neural Comput 14(4):715–770

    Article  MATH  Google Scholar 

  30. Wollmer M, Eyben F, Graves A, Schuller B, Rigoll G (2010) Bidirectional lstm networks for context-sensitive keyword detection in a cognitive virtual agent framework. Cognit Comput 2(3):180

    Article  Google Scholar 

  31. Yang Y, Wu QMJ, Wang Y, Zeeshan KM, Lin X, Yuan X (2014) Data partition learning with multiple extreme learning machines. IEEE Trans Cybern. PP(99):18–20. doi:10.1109/TCYB.2014.2352594

  32. Yger F, Berar M, Gasso G, Rakotomamonjy A (2012) Adaptive canonical correlation analysis based on matrix manifolds. In: Proceedings of the international conference on machine learning. New york, NY, ACM

    Google Scholar 

  33. Zeck G, Bethge M, Macke JH (2008) Receptive fields without spike-triggering. In: Platt JC, Koller D, Singer Y, Roweis ST (eds) Advances in neural information processing systems 20. Curran Associates, Inc, pp 969–976

  34. Zhang Q, Leung YW (2000) A class of learning algorithms for principal component analysis and minor component analysis. IEEE Trans Neural Netw 11(2):200–204

    Article  Google Scholar 

Download references

Acknowledgements

The first author is grateful to Professor Colin Fyfe, formerly with the University of The West of Scotland, for his insightful suggestions which helped improve the writing of this paper. This work of Zeeshan Malik was supported by University of The Punjab, Lahore, Pakistan, as part of the Overseas Scholarship for his Doctoral Studies in the University of Stirling, UK.

Author information

Authors and Affiliations

  1. University of Stirling, Stirling, Scotland, UK

    Zeeshan Khawar Malik & Amir Hussain

  2. University of Windsor, Windsor , ON, Canada

    Q. M. Jonathan Wu

Authors
  1. Zeeshan Khawar Malik
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Amir Hussain
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Q. M. Jonathan Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zeeshan Khawar Malik.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Malik, Z.K., Hussain, A. & Wu, Q.M.J. Extracting online information from dual and multiple data streams. Neural Comput & Applic 30, 87–98 (2018). https://doi.org/10.1007/s00521-016-2647-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-016-2647-3

Keywords

Search

Navigation