Data Mining and Knowledge Discovery

, Volume 30, Issue 5, pp 1112–1133 | Cite as

Using regression makes extraction of shared variation in multiple datasets easy

  • Jussi Korpela
  • Andreas Henelius
  • Lauri Ahonen
  • Arto Klami
  • Kai Puolamäki
Article
  • 447 Downloads

Abstract

In many data analysis tasks it is important to understand the relationships between different datasets. Several methods exist for this task but many of them are limited to two datasets and linear relationships. In this paper, we propose a new efficient algorithm, termed cocoreg, for the extraction of variation common to all datasets in a given collection of arbitrary size. cocoreg extends redundancy analysis to more than two datasets, utilizing chains of regression functions to extract the shared variation in the original data space. The algorithm can be used with any linear or non-linear regression function, which makes it robust, straightforward, fast, and easy to implement and use. We empirically demonstrate the efficacy of shared variation extraction using the cocoreg algorithm on five artificial and three real datasets.

Keywords

Shared variation Multiple regression Regression chains 

References

  1. Andrew G, Arora R, Bilmes J, Livescu K (2013) Deep canonical correlation analysis. In: Proceedings of the 30th international conference on machine learning, vol 28, pp 1247–1255Google Scholar
  2. Dähne S, Nikulin VV, Ramírez D, Schreier PJ, Müller KR, Haufe S (2014) Finding brain oscillations with power dependencies in neuroimaging data. NeuroImage 96:334–348CrossRefGoogle Scholar
  3. Damianou A, Ek C, Titsias MK, Lawrence ND (2012) Manifold relevance determination. In: Proceedings of the 29th international conference on machine learning, pp 145–152Google Scholar
  4. Fisher J, Darrell T (2003) Speaker association with signal-level audiovisual fusion. IEEE Trans Multimed 6(3):406–413CrossRefGoogle Scholar
  5. Hardoon D, Szedmak S, Shawe-Taylor J (2004) Canonical correlation analysis: an overview with application to learning methods. Neural Comput 16(12):2639–2664. doi:10.1162/0899766042321814 CrossRefMATHGoogle Scholar
  6. Hasson U, Nir Y, Levy I, Fuhrmann G, Malach R (2004) Intersubject synchronization of cortical activity during natural vision. Science 303(5664):1634–1640CrossRefGoogle Scholar
  7. Hastie T, Tibshirani R, Friedman J (2003) The elements of statistical learning: data mining, inference, and prediction. Springer, New YorkMATHGoogle Scholar
  8. Hotelling H (1936) Relations between two sets of variates. Biometrika 28:321–377CrossRefMATHGoogle Scholar
  9. Hsieh WW (2000) Nonlinear canonical correlation analysis by neural networks. Neural Netw 13:1095–1105CrossRefGoogle Scholar
  10. Hwang H, Jung K, Takane Y, Woodward TS (2013) A unified approach to multiple-set canonical correlation analysis and principal components analysis. Br J Math Stat Psychol 66(2):308–321. doi:10.1111/j.2044-8317.2012.02052.x MathSciNetCrossRefGoogle Scholar
  11. Kettenring J (1971) Canonical analysis of several sets of variables. Biometrika 58:433–451MathSciNetCrossRefMATHGoogle Scholar
  12. Klami A, Virtanen S, Kaski S (2013) Bayesian canonical correlation analysis. J Mach Learn Res 14:965–1003MathSciNetMATHGoogle Scholar
  13. Klami A, Virtanen S, Leppäho E (2015) Group factor analysis. IEEE Trans Neural Netw Learn Syst 26(9):2136–2147. doi:10.1109/TNNLS.2014.2376974 MathSciNetCrossRefGoogle Scholar
  14. Korpela J, Henelius A (2016) Cocoreg: extracts shared variation in collections of datasets using regression models. http://cran.r-project.org/package=cocoreg
  15. Legendre P, Legendre L (1998) Numerical ecology, 2nd edn. Elsevier, AmsterdamMATHGoogle Scholar
  16. Liaw A, Wiener M (2002) Classification and regression by randomforest. R News 2(3):18–22. https://cran.r-project.org/package=randomForest
  17. Meyer D, Dimitriadou E, Hornik K, Weingessel A, Leisch F (2014) e1071: misc functions of the department of statistics (e1071), Technische Universität Wien. http://cran.r-project.org/package=e1071
  18. Müller KE (1982) Understanding canonical correlation through the general linear model and principal components. Am Stat 36(4):342–354. doi:10.1080/00031305.1982.10483045 MATHGoogle Scholar
  19. Nguyen HV, Müller E, Vreeken J, Efros P, Böhm K (2014) Multivariate maximal correlation analysis. In: Proceedings of the 31st international conference on machine learning, pp 775–783Google Scholar
  20. R Core Team (2014) R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, http://www.R-project.org/
  21. Tenenhaus A (2011) Regularized generalized canonical correlation analysis and PLS path modeling. Psychometrika 76(2):257–284MathSciNetCrossRefMATHGoogle Scholar
  22. Tibshirani R (1996) Regression shrinkage and selection via the lasso. J R Stat Soc Ser B 58(1):267–288MathSciNetMATHGoogle Scholar
  23. Timmerman ME, Kiers H (2003) Four simultaneous component models for the analysis of multivariate time series from more than one subject to model intraindividual and interindividual differences. Psychometrika 68(1):105–121. doi:10.1007/BF02296656 MathSciNetCrossRefMATHGoogle Scholar
  24. Virtanen S, Klami A, Khan SA, Kaski S (2012) CCAGFA: Bayesian canonical correlation analysis and group factor analysis. http://cran.r-project.org/package=CCAGFA

Copyright information

© The Author(s) 2016

Authors and Affiliations

  • Jussi Korpela
    • 1
  • Andreas Henelius
    • 1
  • Lauri Ahonen
    • 1
  • Arto Klami
    • 2
  • Kai Puolamäki
    • 1
  1. 1.Finnish Institute of Occupational HealthHelsinkiFinland
  2. 2.Department of Computer Science, Helsinki Institute for Information Technology HIITUniversity of HelsinkiHelsinkiFinland

Personalised recommendations